-- Logs begin at Wed 2022-08-03 12:13:06 UTC, end at Wed 2022-08-03 13:11:26 UTC. -- Aug 03 12:38:56.086798 dv-cinder-851996-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 03 12:38:56.096509 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 03 12:38:56.098526 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 3 12:38:56 2022] *** Aug 03 12:38:56.098526 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 03 12:38:56.098526 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 03 12:38:56.098526 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: nodename: dv-cinder-851996-1-iscsi Aug 03 12:38:56.098526 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: machine: x86_64 Aug 03 12:38:56.098526 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: clock source: unix Aug 03 12:38:56.098526 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: pcre jit disabled Aug 03 12:38:56.098526 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: detected number of CPU cores: 4 Aug 03 12:38:56.098526 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: current working directory: / Aug 03 12:38:56.098526 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: detected binary path: /usr/bin/uwsgi-core Aug 03 12:38:56.099179 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: your processes number limit is 47680 Aug 03 12:38:56.099179 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: your memory page size is 4096 bytes Aug 03 12:38:56.099179 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: detected max file descriptor number: 2048 Aug 03 12:38:56.099179 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: lock engine: pthread robust mutexes Aug 03 12:38:56.099179 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: thunder lock: enabled Aug 03 12:38:56.099179 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 03 12:38:56.099179 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 03 12:38:56.137485 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: Python main interpreter initialized at 0x557bd36169c0 Aug 03 12:38:56.137485 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: python threads support enabled Aug 03 12:38:56.137485 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: your server socket listen backlog is limited to 100 connections Aug 03 12:38:56.137485 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: your mercy for graceful operations on workers is 90 seconds Aug 03 12:38:56.137773 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: mapped 403077 bytes (393 KB) for 2 cores Aug 03 12:38:56.137837 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: *** Operational MODE: preforking *** Aug 03 12:38:56.138105 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: *** uWSGI is running in multiple interpreter mode *** Aug 03 12:38:56.138105 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: spawned uWSGI master process (pid: 80295) Aug 03 12:38:56.138289 dv-cinder-851996-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 03 12:38:56.139136 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: spawned uWSGI worker 1 (pid: 80305, cores: 1) Aug 03 12:38:56.139136 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: spawned uWSGI worker 2 (pid: 80306, cores: 1) Aug 03 12:38:56.139293 dv-cinder-851996-1-iscsi devstack@keystone.service[80295]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 03 12:39:01.653285 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 03 12:39:01.653285 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: from cryptography.utils import int_from_bytes Aug 03 12:39:01.654120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 03 12:39:01.654120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: from cryptography.utils import int_from_bytes Aug 03 12:39:01.654266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 03 12:39:01.654266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: from cryptography.utils import int_from_bytes Aug 03 12:39:01.654266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 03 12:39:01.654266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: from cryptography.utils import int_from_bytes Aug 03 12:39:01.704060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 03 12:39:01.704248 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 03 12:39:01.704353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 03 12:39:01.704441 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] command line args: [] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 03 12:39:01.704513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 03 12:39:01.704698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 03 12:39:01.704784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] command line args: [] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 03 12:39:01.704867 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ================================================================================ {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 03 12:39:01.704998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 03 12:39:01.705153 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] admin_token = **** {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.705232 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ================================================================================ {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 03 12:39:01.705400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] config_dir = [] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.705478 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] admin_token = **** {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.705589 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.705669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] config_dir = [] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.705773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] config_source = [] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.705855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.705955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] debug = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.706043 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] config_source = [] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.706154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.706401 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] debug = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.706401 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.706590 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.706661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.706867 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.706929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.706987 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.707044 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.707171 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.707272 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.707400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] list_limit = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.707501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] list_limit = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.707600 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] log_config_append = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.707693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] log_config_append = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.707796 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.707896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.708000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] log_dir = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.708094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] log_dir = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.708187 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] log_file = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.708279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] log_file = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.708386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.708507 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.708583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.708695 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.708775 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.708888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.708999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.709080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.709202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.709288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.709365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.709492 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.709575 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.709654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.709809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.709890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.710044 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.710133 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.710280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.710357 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.710532 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.710622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.710769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.710853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.711016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.711102 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.711260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.711342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.711534 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.711625 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.712029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.712108 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.712234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] publish_errors = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.712309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] publish_errors = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.712429 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.712536 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.712656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.712773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.712857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.712954 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.713074 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.713201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.713309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.713437 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.713547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] standard_threads = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.713675 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] standard_threads = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.713781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.713911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.714026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.714139 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.714217 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.714318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.714395 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] use_journal = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.714514 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] use_journal = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.714604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] use_json = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.714683 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] use_json = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.714759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] use_stderr = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.714861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] use_stderr = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.714938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] use_syslog = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.715039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] use_syslog = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.715122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.715272 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 03 12:39:01.715355 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.715530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.715608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.715772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.715848 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.716033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.716110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.716277 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.716358 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.716523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.716605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.716762 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.716843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.717002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.717083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.717242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.717324 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.717487 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.717562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.717733 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.717809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.717967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.718049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.718210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.718292 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.718448 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.718552 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.718722 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.718807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.718970 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.719045 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.719207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.719283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.719432 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.719510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.719691 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.719775 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.719942 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.720018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.720182 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.720260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.720420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.720496 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.721203 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.721282 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.721462 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.721539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.721715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.721792 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.721950 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.722033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.722182 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.722260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.722414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.722521 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.722673 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.722765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.722913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.722989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.723159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.723236 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.723391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.723476 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.723623 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.723700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.723855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.723937 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.724106 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.724195 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.724342 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.724423 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.724582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.724669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.724814 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.724898 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.725052 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.725129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.725288 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.725364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.725521 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.725673 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.725747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.725815 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.725977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.726116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.726190 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.726271 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.726432 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.726565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.726702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.726778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.726935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.727018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.727175 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.727258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.727406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.727489 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.727638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.727721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.727903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.728017 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.728113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.728204 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.728344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.728453 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.728576 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.728683 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.728812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.728919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.729034 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.729117 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.729274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.729351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.729510 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.729589 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.729741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.729826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.729974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.730050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.730214 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.730300 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.730451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.730557 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.730707 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.730795 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.730923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.731007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.731190 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.731272 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.731422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.731509 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.731653 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.731729 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.731887 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.731963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.732123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.732211 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.732457 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.732620 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.732699 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.732864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.732980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.733102 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.733180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.733341 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.733425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.733578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.733670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.733817 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.733896 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.734053 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.734129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.734292 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.734371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.734555 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.734675 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.734795 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.734871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.735033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.735112 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.735268 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.735344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.735504 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.735582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.735741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.735818 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.735979 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.736063 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.736313 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.736897 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.737051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.737130 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.737298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.737377 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.737542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.737619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.737787 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.737868 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.738024 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.738101 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.738261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.738337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.738533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.738627 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.738745 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.738824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.738976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.739061 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.739211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.739289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.739439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.739523 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.739670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.739753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.739908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.739985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.740145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.740222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.740393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.740471 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.740633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.740718 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.740890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.740969 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.741137 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.741259 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.741371 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] auth.external = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.741458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.741604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.741683 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] auth.external = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.741843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.741921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.742120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.742211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.742352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] auth.password = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.742442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.742610 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] auth.token = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.742700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] auth.password = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.742844 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] auth.token = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.742927 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.743082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.743168 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.743320 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.743404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.743553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.743631 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.743792 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.743878 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.744039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.744116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.744274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.744352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.744507 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.744592 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] credential.caching = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.744747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] credential.caching = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.744826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.744995 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.745073 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.745244 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.745350 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.745481 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.745559 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.745725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.745802 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.745961 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.746046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.746199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.746283 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.746464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.746597 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.746727 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.746821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.746974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.747054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.747230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.747310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.747489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.747578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.747751 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.747829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.747998 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.748076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.748249 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.748335 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.748487 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] federation.caching = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.748570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.748726 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.748805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] federation.caching = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.748974 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.749062 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.749222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.749299 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.749452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.749529 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.749695 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.749782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.749930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.750008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.750171 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.750249 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.750410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.750530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.750690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.750768 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.750927 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.751005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.751176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.751266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.751416 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity.caching = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.751496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.751655 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity.caching = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.751733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.751928 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.752009 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.752170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.752249 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.752410 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.752488 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.752656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.752736 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.752890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.752969 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.753127 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.753204 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.753370 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.753456 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.753621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.753701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.753854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.753933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.754088 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.754169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.754322 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.754399 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.754581 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.754664 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.754829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.754909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.755075 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.755152 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.755326 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.755423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.755570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.755654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.755815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.755893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.756065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.756143 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.756303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.756383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.756535 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.756614 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.756769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.756854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.756999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.757077 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.757231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.757308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.757469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.757548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.757708 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.757787 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.757950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.758028 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.758181 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.758259 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.758419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.758523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.758689 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.758768 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.758927 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.759006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.759163 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.759244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.759411 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.759488 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.759643 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.759722 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.759880 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.759959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.760109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.760186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.760340 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.760418 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.760577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.760656 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.760814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.760891 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.761054 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.761133 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.761291 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.761413 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.761529 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.761608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.761776 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.761854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.762009 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.762089 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.762240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.762319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.762472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.762597 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.762751 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.762834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.762988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.763065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.763226 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.763312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.763464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.763542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.763693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.763771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.763924 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.764009 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.764166 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.764245 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.764402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.764479 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.764648 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.764733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.764892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.764970 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.765137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.765215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.765372 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.765458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.765601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.765680 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.765832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.765910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.766065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.766155 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.766301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.766385 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.766559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.766639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.766791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.766879 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.767039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.767118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.767283 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.767360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.768054 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.768143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.768392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.768471 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.768625 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.768702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.768866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.768951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.769105 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.769182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.769342 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.769420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.769579 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.769665 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.769815 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.769894 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.770063 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.770161 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.770312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.770545 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.770627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.770805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.770882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.771043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.771121 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.771291 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.771372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.771523 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.771604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.771760 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.771845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.771999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.772079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.772243 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.772321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.772482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.772559 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.772715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.772800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.772951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.773030 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.773185 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.773263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] resource.caching = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.773426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.773506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.773670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] resource.caching = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.773748 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.773905 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.773983 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.774156 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.774237 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.774394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.774472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.774655 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.774733 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.774903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.774983 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.775140 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.775219 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.775381 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.775459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.775612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] role.caching = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.775688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.775848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.775926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] role.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.776076 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.776154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] role.caching = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.776324 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] role.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.776404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.776558 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.776637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.776807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.776885 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.777053 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.777139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.777298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.777376 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.777545 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.777632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.777792 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.777870 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.778038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.778115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.778272 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.778358 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.778541 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.778623 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.778780 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.778858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.779028 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.779117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.779275 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.779353 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.779524 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.779678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.779810 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.779918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.780169 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.780247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.780421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.780501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.780670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.780757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.780921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.781000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.781156 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.781235 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.781408 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.781494 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.781643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.781721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.781892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.781969 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.782124 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.782211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.782355 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.782435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.782613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.782693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.782851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.782930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.783647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.783739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.783904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.783983 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.784145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.784226 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.784382 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.784461 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.784615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.784694 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.784854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] token.caching = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.784935 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.785092 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.785172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.785331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] token.caching = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.785410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.785569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.785648 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.785818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.785895 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.786060 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.786138 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.786295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.786374 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.786565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.786649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.786809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.786887 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.787056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.787136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.787294 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.787373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.787533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.787612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.787769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.787850 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.788007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.788086 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.788247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.788326 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.788474 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.788553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.788719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.788798 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.788950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.789028 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.789198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.789278 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.connection = **** {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.789494 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.789581 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.789745 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.789825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.connection = **** {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.789984 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.790072 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.790227 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.790319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.790467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.790596 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.790727 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.790805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.790965 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.791056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.791201 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.791279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.791438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.791515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.791677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.791766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.791921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.792001 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.792157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.792235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.792404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.792491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.792640 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.792718 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.792892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.793014 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.793117 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.793216 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.793362 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.793454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.793595 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.793683 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.793758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80306) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 03 12:39:01.793842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.794067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 03 12:39:01.794239 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80305) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 03 12:39:01.823741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.823976 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.824204 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.824348 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.824590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.824806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.832531 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.833180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.862403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.862659 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.863083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.863317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.915288 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.915663 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.966186 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.966423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.967144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:01.967383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.030933 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.031169 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.031265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.031397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.031505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.031615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.031703 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.031838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.031926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.032052 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.032149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.032265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.055644 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.055881 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.055974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.056110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.056194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.056338 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.056422 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.056559 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.056651 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.056775 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.056856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.056990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.057079 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.057360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.068839 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.069120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.081446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.081679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.081903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.082081 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.082166 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.082315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.082398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.082560 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.082658 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.082777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80306) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.082860 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.082993 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.083209 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.083423 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80305) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 03 12:39:02.099511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.099721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 03 12:39:02.100056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.100138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.100258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 03 12:39:02.100356 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 03 12:39:02.100525 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.100671 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.100781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.100864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 03 12:39:02.101038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.101121 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.101292 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.101373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.101621 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.101879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.110915 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.111715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.111985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.112108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.112262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.112385 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.112650 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.112926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.119150 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.119876 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.123781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.124063 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.124342 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.124560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.124644 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.124843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.124940 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.125127 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.125210 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.125392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.125472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.125662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.125745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.125928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.126008 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.126196 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.126276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.126462 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.126565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.126764 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.127584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.127854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.149766 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.150752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.151289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.151568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.152244 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.152536 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.156161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.156439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.156720 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.157122 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.157404 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.157686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.163810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.164091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.164800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.165090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.168336 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.168583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.168880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.169161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.169242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.169435 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.169518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.169727 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.169809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.169996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.170086 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.170270 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.170351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.170595 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.170872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.171155 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.186998 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.187284 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.188103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.188383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.191241 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.192371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.194381 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.194696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.195537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.195835 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.199077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.200238 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.202217 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.203494 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.205425 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.205719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.206012 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.206304 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.206616 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.206743 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.206902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.207042 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.207332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.207622 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.207911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.208198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.223314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.223593 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.223857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.224118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.224687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.224972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.225242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.225504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.231016 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.232438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.232607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.232809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.233973 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.234251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.235648 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.235927 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.236771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.237061 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.237347 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.237707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.237990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.238290 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.238600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.238890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.249304 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.249583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.249864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.250141 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.250417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.250713 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.250984 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.252597 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.252880 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.253162 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.253443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.253724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.254001 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.254276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.266433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.269422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.269834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.272379 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.272822 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.274858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.275913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.276356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.276639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.276921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.278397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.279925 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.280214 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.280499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.283954 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.285554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.287502 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.288489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.288779 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.289061 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.289144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.289345 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.292018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.292311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.292598 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.292892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.297349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.297636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.297925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.301515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.301801 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.302095 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.305364 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80306) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.305641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.305918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.306190 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.306461 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.306762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.307042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.307313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.307681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.307971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.308250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.308512 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.308799 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.308991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80305) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 03 12:39:02.309082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80306) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.309274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.309558 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.309838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.310393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.310691 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.310978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.311258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.311629 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.311914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.312195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.312471 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.312757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.313022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80305) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 03 12:39:02.342388 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557bd36169c0 pid: 80306 (default app) Aug 03 12:39:02.346220 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580309cc-6622-47de-abed-e4975cf4f63f None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:02.346528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580309cc-6622-47de-abed-e4975cf4f63f None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:02.346809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580309cc-6622-47de-abed-e4975cf4f63f None None] PATH_INFO: `/v3` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:02.346892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557bd36169c0 pid: 80305 (default app) Aug 03 12:39:02.351143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1/1] 10.222.0.65 () {50 vars in 791 bytes} [Wed Aug 3 12:39:02 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:09.893621 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e306b-92cb-4231-95a5-abc8f24dcd93 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:09.894105 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e306b-92cb-4231-95a5-abc8f24dcd93 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:09.894195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e306b-92cb-4231-95a5-abc8f24dcd93 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:09.898639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1/2] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:09.903806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fda10f-6ddb-47db-a0c6-b5bf9ba427f9 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:09.904229 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fda10f-6ddb-47db-a0c6-b5bf9ba427f9 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:09.904361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fda10f-6ddb-47db-a0c6-b5bf9ba427f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:09.975054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: INFO dbcounter [None req-45fda10f-6ddb-47db-a0c6-b5bf9ba427f9 None None] Registered counter for database keystone Aug 03 12:39:09.993857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG oslo_db.sqlalchemy.engines [None req-45fda10f-6ddb-47db-a0c6-b5bf9ba427f9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80306) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 03 12:39:09.999965 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writer thread running {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 03 12:39:10.179455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG passlib.handlers.bcrypt [None req-45fda10f-6ddb-47db-a0c6-b5bf9ba427f9 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80306) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 03 12:39:10.179794 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG passlib.handlers.bcrypt [None req-45fda10f-6ddb-47db-a0c6-b5bf9ba427f9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80306) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 03 12:39:10.233741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-45fda10f-6ddb-47db-a0c6-b5bf9ba427f9 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:10.235315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-45fda10f-6ddb-47db-a0c6-b5bf9ba427f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:10.363920 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 2/3] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 03 12:39:10.502360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e306b-92cb-4231-95a5-abc8f24dcd93 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:10.575408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: INFO dbcounter [None req-563e306b-92cb-4231-95a5-abc8f24dcd93 None None] Registered counter for database keystone Aug 03 12:39:10.596445 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG oslo_db.sqlalchemy.engines [None req-563e306b-92cb-4231-95a5-abc8f24dcd93 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80305) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 03 12:39:10.602496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writer thread running {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 03 12:39:10.720247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e306b-92cb-4231-95a5-abc8f24dcd93 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:10.725166 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ea2c1-8ecb-44b6-b87d-506305aa8ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:10.725799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ea2c1-8ecb-44b6-b87d-506305aa8ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:10.726076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ea2c1-8ecb-44b6-b87d-506305aa8ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:10.726344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ea2c1-8ecb-44b6-b87d-506305aa8ff8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:10.734341 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ea2c1-8ecb-44b6-b87d-506305aa8ff8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:10.786198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: WARNING keystone.common.rbac_enforcer.enforcer [None req-032ea2c1-8ecb-44b6-b87d-506305aa8ff8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 03 12:39:10.786512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ea2c1-8ecb-44b6-b87d-506305aa8ff8 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:10.795861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 2/4] 10.222.0.65 () {58 vars in 1324 bytes} [Wed Aug 3 12:39:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 12:39:10.801785 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fda10f-6ddb-47db-a0c6-b5bf9ba427f9 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:10.816377 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fda10f-6ddb-47db-a0c6-b5bf9ba427f9 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:10.820797 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fbfb51-bc9c-4bb5-8696-5ebad52e091b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:10.821929 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbfb51-bc9c-4bb5-8696-5ebad52e091b admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:10.822226 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbfb51-bc9c-4bb5-8696-5ebad52e091b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:10.822541 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fbfb51-bc9c-4bb5-8696-5ebad52e091b admin admin] PATH_INFO: `/v3/projects/87eaffc3c69e48b2bb1ac115a06dc966` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:10.824030 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fbfb51-bc9c-4bb5-8696-5ebad52e091b admin admin] RBAC: Authorizing `identity:get_project(project_id=87eaffc3c69e48b2bb1ac115a06dc966)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:10.874212 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f5fbfb51-bc9c-4bb5-8696-5ebad52e091b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 03 12:39:10.874593 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fbfb51-bc9c-4bb5-8696-5ebad52e091b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:10.879287 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 3/5] 10.222.0.65 () {56 vars in 1207 bytes} [Wed Aug 3 12:39:10 2022] GET /identity/v3/projects/87eaffc3c69e48b2bb1ac115a06dc966 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:12.837977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a7c5b-e74a-4196-bddd-faa7d1fa0070 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:12.838577 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a7c5b-e74a-4196-bddd-faa7d1fa0070 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:12.838577 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a7c5b-e74a-4196-bddd-faa7d1fa0070 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:12.842626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 3/6] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:39:12.848036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3925f7-e43e-4a2f-95ac-7cd07071cadc None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:12.848451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3925f7-e43e-4a2f-95ac-7cd07071cadc None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:12.848594 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3925f7-e43e-4a2f-95ac-7cd07071cadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:12.868309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-fc3925f7-e43e-4a2f-95ac-7cd07071cadc None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:12.869825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-fc3925f7-e43e-4a2f-95ac-7cd07071cadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:12.878289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 4/7] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 03 12:39:12.905158 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a7c5b-e74a-4196-bddd-faa7d1fa0070 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:12.917019 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a7c5b-e74a-4196-bddd-faa7d1fa0070 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:12.921146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c7972-8881-4b45-adc1-e7381d79cc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:12.921745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c7972-8881-4b45-adc1-e7381d79cc51 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:12.922023 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c7972-8881-4b45-adc1-e7381d79cc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:12.922294 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c7972-8881-4b45-adc1-e7381d79cc51 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:12.929869 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c7972-8881-4b45-adc1-e7381d79cc51 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:12.931657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c7972-8881-4b45-adc1-e7381d79cc51 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:12.941026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 4/8] 10.222.0.65 () {58 vars in 1324 bytes} [Wed Aug 3 12:39:12 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 12:39:12.946312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3925f7-e43e-4a2f-95ac-7cd07071cadc None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:12.956018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3925f7-e43e-4a2f-95ac-7cd07071cadc None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:12.960516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220aa56-4651-400d-9bad-62771df0bb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:12.961149 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220aa56-4651-400d-9bad-62771df0bb39 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:12.961436 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220aa56-4651-400d-9bad-62771df0bb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:12.961715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220aa56-4651-400d-9bad-62771df0bb39 admin admin] PATH_INFO: `/v3/users/f665e42501ed48eda8620c2548069351` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:12.963211 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220aa56-4651-400d-9bad-62771df0bb39 admin admin] RBAC: Authorizing `identity:get_user(user_id=f665e42501ed48eda8620c2548069351)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:12.965039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220aa56-4651-400d-9bad-62771df0bb39 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:12.966629 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 5/9] 10.222.0.65 () {56 vars in 1198 bytes} [Wed Aug 3 12:39:12 2022] GET /identity/v3/users/f665e42501ed48eda8620c2548069351 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:15.025849 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32476dc5-538a-4cf0-be0f-48d84dbd84db None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:15.026393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d7197-4748-4963-8d8e-ebd8cf75745e None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:15.026771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32476dc5-538a-4cf0-be0f-48d84dbd84db None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:15.026856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d7197-4748-4963-8d8e-ebd8cf75745e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:15.026951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32476dc5-538a-4cf0-be0f-48d84dbd84db None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:15.027044 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d7197-4748-4963-8d8e-ebd8cf75745e None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:15.027761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 5/10] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:15.028198 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 6/11] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:39:15.032616 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de659695-7b78-4be6-bd59-6f0e385494fa None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:15.033355 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca0233-1062-4097-9b99-fb74205ac78d None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:15.033468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de659695-7b78-4be6-bd59-6f0e385494fa None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:15.033645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca0233-1062-4097-9b99-fb74205ac78d None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:15.033731 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de659695-7b78-4be6-bd59-6f0e385494fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:15.033922 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca0233-1062-4097-9b99-fb74205ac78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:15.053662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-dcca0233-1062-4097-9b99-fb74205ac78d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:15.055231 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-dcca0233-1062-4097-9b99-fb74205ac78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:15.076169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG passlib.handlers.bcrypt [None req-de659695-7b78-4be6-bd59-6f0e385494fa None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80305) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 03 12:39:15.076607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG passlib.handlers.bcrypt [None req-de659695-7b78-4be6-bd59-6f0e385494fa None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80305) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 03 12:39:15.122829 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-de659695-7b78-4be6-bd59-6f0e385494fa None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:15.124436 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-de659695-7b78-4be6-bd59-6f0e385494fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:15.133507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 7/12] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:39:15.154410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca0233-1062-4097-9b99-fb74205ac78d None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:15.178274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:15.224225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 6/13] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:39:15.247397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de659695-7b78-4be6-bd59-6f0e385494fa None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:15.258072 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca0233-1062-4097-9b99-fb74205ac78d None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:15.293519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb7d0f-eb69-45af-8b64-289264b4d07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:15.294200 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb7d0f-eb69-45af-8b64-289264b4d07f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:15.294499 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb7d0f-eb69-45af-8b64-289264b4d07f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:15.294784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb7d0f-eb69-45af-8b64-289264b4d07f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:15.347566 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de659695-7b78-4be6-bd59-6f0e385494fa None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:15.350604 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:15.360401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb7d0f-eb69-45af-8b64-289264b4d07f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:15.361897 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb7d0f-eb69-45af-8b64-289264b4d07f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:15.384402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:15.385864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1336a-db89-47cc-8870-d4d66653448c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:15.387091 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1336a-db89-47cc-8870-d4d66653448c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:15.387376 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1336a-db89-47cc-8870-d4d66653448c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:15.387650 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1336a-db89-47cc-8870-d4d66653448c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:15.399630 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b1336a-db89-47cc-8870-d4d66653448c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:15.408696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-b4b1336a-db89-47cc-8870-d4d66653448c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:15.408696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:15.408696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:15.408696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:15.408696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:15.408696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:15.408696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:15.408696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:15.408696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:15.408696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:15.408696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:15.408696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:15.408696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:15.408696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:15.408696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:15.408696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:15.409788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:15.409788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:15.409788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:15.409788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:15.409788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:15.409788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:15.409788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:15.409788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:15.409788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:15.409788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:15.409788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:15.409788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:15.409788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:15.409788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:15.409788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:15.409788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:15.410652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:15.410652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:15.410652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:15.410652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:15.410652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:15.410652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:15.410652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:15.410652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:15.410652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:15.410652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:15.410652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:15.410652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:39:15.410652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 7/14] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:39:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:39:15.415756 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1336a-db89-47cc-8870-d4d66653448c None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:15.460007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 8/15] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 12:39:15.465516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb7d0f-eb69-45af-8b64-289264b4d07f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:15.513637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1336a-db89-47cc-8870-d4d66653448c None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:15.548483 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ed5eb-1645-4f06-851a-bc70bdad47b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:15.549098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ed5eb-1645-4f06-851a-bc70bdad47b7 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:15.549383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ed5eb-1645-4f06-851a-bc70bdad47b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:15.549662 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ed5eb-1645-4f06-851a-bc70bdad47b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:15.550387 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ed5eb-1645-4f06-851a-bc70bdad47b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:15.550725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ed5eb-1645-4f06-851a-bc70bdad47b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:15.552163 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ed5eb-1645-4f06-851a-bc70bdad47b7 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:15.560592 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb7d0f-eb69-45af-8b64-289264b4d07f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:15.565713 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 8/16] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:15.571223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ed5eb-1645-4f06-851a-bc70bdad47b7 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:15.583242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:15.595096 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95bf6c-cc3b-4766-9867-7b2a8a1288f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:15.596285 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95bf6c-cc3b-4766-9867-7b2a8a1288f2 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:15.596566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95bf6c-cc3b-4766-9867-7b2a8a1288f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:15.596842 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e95bf6c-cc3b-4766-9867-7b2a8a1288f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:15.597871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:15.606985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95bf6c-cc3b-4766-9867-7b2a8a1288f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:15.608452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e95bf6c-cc3b-4766-9867-7b2a8a1288f2 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:15.626258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-7e95bf6c-cc3b-4766-9867-7b2a8a1288f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:39:15.626258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:15.626258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:15.626258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:15.626258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:15.626258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:15.626258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:15.626258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:15.626258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:15.626258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:15.626258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:15.626258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:15.626258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:39:15.626258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:39:15.626258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:15.627223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:15.628021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:39:15.628021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:39:15.628021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:39:15.628021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:15.628021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 9/17] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:39:15.633197 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95bf6c-cc3b-4766-9867-7b2a8a1288f2 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:15.671122 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ed5eb-1645-4f06-851a-bc70bdad47b7 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:15.704279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818212d-6e9e-4417-b9cb-17397b7f830e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:15.705438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818212d-6e9e-4417-b9cb-17397b7f830e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:15.705714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818212d-6e9e-4417-b9cb-17397b7f830e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:15.705985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818212d-6e9e-4417-b9cb-17397b7f830e None admin] PATH_INFO: `/v3/users/f665e42501ed48eda8620c2548069351` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:15.707534 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818212d-6e9e-4417-b9cb-17397b7f830e None admin] RBAC: Authorizing `identity:get_user(user_id=f665e42501ed48eda8620c2548069351)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:15.709005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818212d-6e9e-4417-b9cb-17397b7f830e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:15.710706 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 9/18] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:15 2022] GET /identity/v3/users/f665e42501ed48eda8620c2548069351 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:15.716359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818212d-6e9e-4417-b9cb-17397b7f830e None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:15.812423 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:15.820688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818212d-6e9e-4417-b9cb-17397b7f830e None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:15.837524 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e95bf6c-cc3b-4766-9867-7b2a8a1288f2 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:15.854746 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18553e92-a356-45d0-9213-2184a4604b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:15.855384 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18553e92-a356-45d0-9213-2184a4604b11 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:15.855669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18553e92-a356-45d0-9213-2184a4604b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:15.855948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18553e92-a356-45d0-9213-2184a4604b11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:15.859405 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18553e92-a356-45d0-9213-2184a4604b11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:15.860862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18553e92-a356-45d0-9213-2184a4604b11 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:15.862623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 10/19] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:15.868135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18553e92-a356-45d0-9213-2184a4604b11 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:15.872569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9e7dd-75f8-4c1e-bb71-a534a755d504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:15.873199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9e7dd-75f8-4c1e-bb71-a534a755d504 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:15.873482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9e7dd-75f8-4c1e-bb71-a534a755d504 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:15.873755 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9e7dd-75f8-4c1e-bb71-a534a755d504 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:15.874460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9e7dd-75f8-4c1e-bb71-a534a755d504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:15.874804 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9e7dd-75f8-4c1e-bb71-a534a755d504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:15.876194 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9e7dd-75f8-4c1e-bb71-a534a755d504 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:15.894932 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 10/20] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:15.971290 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18553e92-a356-45d0-9213-2184a4604b11 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:16.005861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a1cb1-b19b-4ca4-aedd-466e09f3c248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:16.006514 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a1cb1-b19b-4ca4-aedd-466e09f3c248 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:16.006802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a1cb1-b19b-4ca4-aedd-466e09f3c248 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:16.007073 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a1cb1-b19b-4ca4-aedd-466e09f3c248 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:16.007881 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a1cb1-b19b-4ca4-aedd-466e09f3c248 None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.domain.id=default, user.id=f665e42501ed48eda8620c2548069351) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:16.008186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a1cb1-b19b-4ca4-aedd-466e09f3c248 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:16.009634 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45a1cb1-b19b-4ca4-aedd-466e09f3c248 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:16.019120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 11/21] 10.222.0.65 () {56 vars in 1322 bytes} [Wed Aug 3 12:39:15 2022] GET /identity/v3/role_assignments?user.id=f665e42501ed48eda8620c2548069351&scope.domain.id=default&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:18.174630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305c8ff-f703-4d88-a15d-a4da6aff9207 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:18.175587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305c8ff-f703-4d88-a15d-a4da6aff9207 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:18.176066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305c8ff-f703-4d88-a15d-a4da6aff9207 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:18.177941 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 11/22] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:18.183076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c743d-8aa6-4f6c-8310-2b1d61acbbcc None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:18.183907 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c743d-8aa6-4f6c-8310-2b1d61acbbcc None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:18.184411 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3c743d-8aa6-4f6c-8310-2b1d61acbbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:18.211568 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-7e3c743d-8aa6-4f6c-8310-2b1d61acbbcc None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:18.213145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-7e3c743d-8aa6-4f6c-8310-2b1d61acbbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:18.236623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:18.301628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 12/23] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:18.348046 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305c8ff-f703-4d88-a15d-a4da6aff9207 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:18.392274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:18.475308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305c8ff-f703-4d88-a15d-a4da6aff9207 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:18.512577 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf7541-250f-4e53-9b6b-62326e51a9f9 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:18.512906 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf7541-250f-4e53-9b6b-62326e51a9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:18.513141 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf7541-250f-4e53-9b6b-62326e51a9f9 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:18.514505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 13/24] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:39:18.529974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f3463-5f18-4670-86c0-bc106891aac6 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:18.530257 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f3463-5f18-4670-86c0-bc106891aac6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:18.530562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f3463-5f18-4670-86c0-bc106891aac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:18.538917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab9b33-055d-437f-8eec-26e1dac13d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:18.539572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab9b33-055d-437f-8eec-26e1dac13d92 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:18.539851 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab9b33-055d-437f-8eec-26e1dac13d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:18.544749 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab9b33-055d-437f-8eec-26e1dac13d92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:18.554721 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-879f3463-5f18-4670-86c0-bc106891aac6 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:18.556121 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-879f3463-5f18-4670-86c0-bc106891aac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:18.564955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab9b33-055d-437f-8eec-26e1dac13d92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:18.566436 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab9b33-055d-437f-8eec-26e1dac13d92 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:18.594786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-26ab9b33-055d-437f-8eec-26e1dac13d92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:39:18.594786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:18.594786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:18.594786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:18.594786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:18.594786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:18.594786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:18.594786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:18.594786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:18.594786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:18.594786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:18.594786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:18.594786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:39:18.594786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:39:18.594786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:18.595813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:18.596627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:39:18.596627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:39:18.596627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:39:18.596627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:18.596627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 12/25] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:18.603717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8972ab2-b4b8-42bf-a68d-13d0c939cbd7 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:18.604002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8972ab2-b4b8-42bf-a68d-13d0c939cbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:18.604280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8972ab2-b4b8-42bf-a68d-13d0c939cbd7 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:18.605578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 13/26] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:18.608807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8972ab2-b4b8-42bf-a68d-13d0c939cbd7 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:18.684359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 14/27] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 03 12:39:18.686406 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006bacb-9287-4ed7-9b93-075245c8b858 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:18.687012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006bacb-9287-4ed7-9b93-075245c8b858 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:18.687430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006bacb-9287-4ed7-9b93-075245c8b858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:18.777715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:18.831260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8972ab2-b4b8-42bf-a68d-13d0c939cbd7 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:18.840668 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1006bacb-9287-4ed7-9b93-075245c8b858 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:18.843851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1006bacb-9287-4ed7-9b93-075245c8b858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:18.903654 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e55d7-a1a2-4e8f-a7b8-c86c48d46816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:18.904285 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e55d7-a1a2-4e8f-a7b8-c86c48d46816 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:18.904570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e55d7-a1a2-4e8f-a7b8-c86c48d46816 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:18.904847 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e55d7-a1a2-4e8f-a7b8-c86c48d46816 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:18.905556 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e55d7-a1a2-4e8f-a7b8-c86c48d46816 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:18.905866 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e55d7-a1a2-4e8f-a7b8-c86c48d46816 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:18.907287 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e55d7-a1a2-4e8f-a7b8-c86c48d46816 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:18.921525 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 14/28] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:18.922637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e55d7-a1a2-4e8f-a7b8-c86c48d46816 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:18.963364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 15/29] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:18.965266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a363b2b-26fe-4d51-a53c-7428c7968f3e None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:18.965689 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a363b2b-26fe-4d51-a53c-7428c7968f3e None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:18.966086 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a363b2b-26fe-4d51-a53c-7428c7968f3e None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:18.967539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 16/30] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:18.968354 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a363b2b-26fe-4d51-a53c-7428c7968f3e None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:18.990188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:19.053018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e55d7-a1a2-4e8f-a7b8-c86c48d46816 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:19.097818 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e4adad-bc96-43dc-ac3f-5d2a55a30a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:19.098507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4adad-bc96-43dc-ac3f-5d2a55a30a8d None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:19.098787 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4adad-bc96-43dc-ac3f-5d2a55a30a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:19.099231 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e4adad-bc96-43dc-ac3f-5d2a55a30a8d None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:19.100840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4adad-bc96-43dc-ac3f-5d2a55a30a8d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:19.102833 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e4adad-bc96-43dc-ac3f-5d2a55a30a8d None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:19.104843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a363b2b-26fe-4d51-a53c-7428c7968f3e None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:19.147892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf2fd6-a097-4afe-b1e6-e93d5e275e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:19.148915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf2fd6-a097-4afe-b1e6-e93d5e275e46 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:19.149452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf2fd6-a097-4afe-b1e6-e93d5e275e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:19.149953 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf2fd6-a097-4afe-b1e6-e93d5e275e46 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:19.150960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf2fd6-a097-4afe-b1e6-e93d5e275e46 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:19.152649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf2fd6-a097-4afe-b1e6-e93d5e275e46 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:19.239482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 15/31] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 3 12:39:18 2022] POST /identity/v3/roles => generated 219 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:39:19.241182 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76943c3-0ea3-42ab-880e-72fdac7e8c76 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:19.241535 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76943c3-0ea3-42ab-880e-72fdac7e8c76 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:19.241784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76943c3-0ea3-42ab-880e-72fdac7e8c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:19.255822 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:19.265131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a76943c3-0ea3-42ab-880e-72fdac7e8c76 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:19.266662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a76943c3-0ea3-42ab-880e-72fdac7e8c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:19.327637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 17/32] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 3 12:39:18 2022] POST /identity/v3/projects => generated 301 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:39:19.329535 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf2fd6-a097-4afe-b1e6-e93d5e275e46 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:19.378578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 16/33] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:19.422401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76943c3-0ea3-42ab-880e-72fdac7e8c76 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:19.481721 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:19.529972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf2fd6-a097-4afe-b1e6-e93d5e275e46 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:19.579675 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c914ef-3d7d-4349-a409-d65cb9f50cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:19.585107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c914ef-3d7d-4349-a409-d65cb9f50cb7 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:19.585400 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c914ef-3d7d-4349-a409-d65cb9f50cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:19.585676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c914ef-3d7d-4349-a409-d65cb9f50cb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:19.586476 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c914ef-3d7d-4349-a409-d65cb9f50cb7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:19.588363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c914ef-3d7d-4349-a409-d65cb9f50cb7 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:19.613456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 18/34] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 3 12:39:19 2022] POST /identity/v3/roles => generated 225 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:39:19.648842 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76943c3-0ea3-42ab-880e-72fdac7e8c76 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:19.689644 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a268a-6bbf-4542-a9a5-60a74f25cc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:19.690367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a268a-6bbf-4542-a9a5-60a74f25cc62 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:19.690684 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a268a-6bbf-4542-a9a5-60a74f25cc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:19.690964 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a268a-6bbf-4542-a9a5-60a74f25cc62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:19.695366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:19.706304 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a268a-6bbf-4542-a9a5-60a74f25cc62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:19.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-085a268a-6bbf-4542-a9a5-60a74f25cc62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:19.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:19.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:19.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:19.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:19.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:19.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:19.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:19.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:19.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:19.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:19.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:19.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:19.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:19.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:19.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:19.718662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:19.718662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:19.718662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:19.718662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:19.718662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:19.718662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:19.718662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:19.718662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:19.718662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:19.718662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:19.718662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:19.718662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:19.718662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:19.718662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:19.718662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:19.718662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:19.719621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:19.719621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:19.719621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:19.719621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:19.719621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:19.719621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:19.719621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:19.719621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:19.719621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:19.719621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:19.719621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:19.719621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:19.722102 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 17/35] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:39:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:19.728405 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c914ef-3d7d-4349-a409-d65cb9f50cb7 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:19.803813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be77a3-0210-4c00-b748-7242a8827282 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:19.804318 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be77a3-0210-4c00-b748-7242a8827282 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:19.804780 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53be77a3-0210-4c00-b748-7242a8827282 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:19.806558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 18/36] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:39:19.813213 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2417f45-d4d0-489b-a4a3-c9d8902df3fb None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:19.813777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2417f45-d4d0-489b-a4a3-c9d8902df3fb None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:19.814897 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2417f45-d4d0-489b-a4a3-c9d8902df3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:19.834577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-f2417f45-d4d0-489b-a4a3-c9d8902df3fb None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:19.835865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f2417f45-d4d0-489b-a4a3-c9d8902df3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:19.847982 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c914ef-3d7d-4349-a409-d65cb9f50cb7 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:19.880405 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:19.889816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd5ef0-9dee-478a-97e9-c7e7a0b4c3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:19.890443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd5ef0-9dee-478a-97e9-c7e7a0b4c3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:19.890760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd5ef0-9dee-478a-97e9-c7e7a0b4c3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:19.891031 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd5ef0-9dee-478a-97e9-c7e7a0b4c3c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:19.891776 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd5ef0-9dee-478a-97e9-c7e7a0b4c3c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:19.892080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd5ef0-9dee-478a-97e9-c7e7a0b4c3c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:19.893552 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd5ef0-9dee-478a-97e9-c7e7a0b4c3c8 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:19.901850 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 19/37] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:19.903290 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c75857-3ccf-4c29-9dea-c66c5606c306 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:19.903578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c75857-3ccf-4c29-9dea-c66c5606c306 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:19.903855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c75857-3ccf-4c29-9dea-c66c5606c306 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:19.905062 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 20/38] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:19.907385 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c75857-3ccf-4c29-9dea-c66c5606c306 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:19.919987 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 19/39] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:39:19.921307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4056a-0721-4094-9bef-89cc14414878 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:19.921607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4056a-0721-4094-9bef-89cc14414878 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:19.921885 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b4056a-0721-4094-9bef-89cc14414878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:19.946282 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-f6b4056a-0721-4094-9bef-89cc14414878 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:19.947515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f6b4056a-0721-4094-9bef-89cc14414878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:20.012133 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c75857-3ccf-4c29-9dea-c66c5606c306 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:20.013203 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 20/40] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:20.014089 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4056a-0721-4094-9bef-89cc14414878 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:20.047310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40d8c3-73aa-4757-a70a-786b751e0913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:20.047950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40d8c3-73aa-4757-a70a-786b751e0913 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:20.048231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40d8c3-73aa-4757-a70a-786b751e0913 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:20.049036 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40d8c3-73aa-4757-a70a-786b751e0913 None admin] PATH_INFO: `/v3/users/f665e42501ed48eda8620c2548069351` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:20.050216 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40d8c3-73aa-4757-a70a-786b751e0913 None admin] RBAC: Authorizing `identity:get_user(user_id=f665e42501ed48eda8620c2548069351)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:20.051707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40d8c3-73aa-4757-a70a-786b751e0913 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:20.053304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 21/41] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:19 2022] GET /identity/v3/users/f665e42501ed48eda8620c2548069351 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:20.054053 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40d8c3-73aa-4757-a70a-786b751e0913 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:20.069260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:20.111578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:20.112427 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b4056a-0721-4094-9bef-89cc14414878 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:20.146129 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99463bb-428d-4330-a4ae-c2f9ef1b6455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:20.146780 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99463bb-428d-4330-a4ae-c2f9ef1b6455 None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:20.147069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99463bb-428d-4330-a4ae-c2f9ef1b6455 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:20.147348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99463bb-428d-4330-a4ae-c2f9ef1b6455 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:20.148059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99463bb-428d-4330-a4ae-c2f9ef1b6455 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:20.149569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99463bb-428d-4330-a4ae-c2f9ef1b6455 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:20.153136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40d8c3-73aa-4757-a70a-786b751e0913 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:20.164285 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 21/42] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 3 12:39:20 2022] POST /identity/v3/roles => generated 223 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:39:20.165159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99463bb-428d-4330-a4ae-c2f9ef1b6455 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:20.194025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020ea33-6532-4593-a203-413cef6fbd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:20.194832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020ea33-6532-4593-a203-413cef6fbd7f None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:20.195116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020ea33-6532-4593-a203-413cef6fbd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:20.195449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020ea33-6532-4593-a203-413cef6fbd7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:20.196726 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020ea33-6532-4593-a203-413cef6fbd7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:20.198221 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020ea33-6532-4593-a203-413cef6fbd7f None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:20.200126 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 22/43] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:20.205678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020ea33-6532-4593-a203-413cef6fbd7f None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:20.280689 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99463bb-428d-4330-a4ae-c2f9ef1b6455 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:20.304094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:20.319650 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020ea33-6532-4593-a203-413cef6fbd7f None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:20.331519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfa2b0-36ab-44a7-b6e1-3d1847c6105f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:20.332406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfa2b0-36ab-44a7-b6e1-3d1847c6105f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:20.332697 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfa2b0-36ab-44a7-b6e1-3d1847c6105f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:20.332956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cfa2b0-36ab-44a7-b6e1-3d1847c6105f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:20.334641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfa2b0-36ab-44a7-b6e1-3d1847c6105f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:20.336622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cfa2b0-36ab-44a7-b6e1-3d1847c6105f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:20.339606 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 22/44] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:20.344475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfa2b0-36ab-44a7-b6e1-3d1847c6105f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:20.352219 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:20.363751 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f1890-1604-4e15-b750-13ba45e2b2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:20.364317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f1890-1604-4e15-b750-13ba45e2b2bd None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:20.364601 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f1890-1604-4e15-b750-13ba45e2b2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:20.364879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f1890-1604-4e15-b750-13ba45e2b2bd None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:20.365573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f1890-1604-4e15-b750-13ba45e2b2bd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:20.367046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f1890-1604-4e15-b750-13ba45e2b2bd None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:20.393784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 23/45] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 3 12:39:20 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:39:20.451029 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cfa2b0-36ab-44a7-b6e1-3d1847c6105f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:20.486782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612454d-5330-4cca-936d-e8b4a5a2909b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:20.487477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612454d-5330-4cca-936d-e8b4a5a2909b None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:20.487759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612454d-5330-4cca-936d-e8b4a5a2909b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:20.488037 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612454d-5330-4cca-936d-e8b4a5a2909b None admin] PATH_INFO: `/v3/domains/default/users/f665e42501ed48eda8620c2548069351/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:20.490078 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612454d-5330-4cca-936d-e8b4a5a2909b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f665e42501ed48eda8620c2548069351, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:20.491598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612454d-5330-4cca-936d-e8b4a5a2909b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:20.505967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 23/46] 10.222.0.65 () {58 vars in 1361 bytes} [Wed Aug 3 12:39:20 2022] PUT /identity/v3/domains/default/users/f665e42501ed48eda8620c2548069351/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:39:22.372938 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f2942-2bb0-4ba8-9947-dc0ec941c85e None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:22.373454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f2942-2bb0-4ba8-9947-dc0ec941c85e None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:22.373538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f2942-2bb0-4ba8-9947-dc0ec941c85e None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:22.374858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 24/47] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:22.380020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed3598-d8be-44ea-b4ea-0b7035c6836f None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:22.380421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed3598-d8be-44ea-b4ea-0b7035c6836f None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:22.380705 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed3598-d8be-44ea-b4ea-0b7035c6836f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:22.399033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-e4ed3598-d8be-44ea-b4ea-0b7035c6836f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:22.400221 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e4ed3598-d8be-44ea-b4ea-0b7035c6836f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:22.439948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:22.466628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 24/48] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:39:22.469228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4fd850-201b-4b95-a69f-edd70e9fa34f None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:22.469515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4fd850-201b-4b95-a69f-edd70e9fa34f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:22.469789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4fd850-201b-4b95-a69f-edd70e9fa34f None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:22.471151 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 25/49] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:22.476079 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0b660-e9b8-45a1-a9f7-2848461f5c7f None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:22.476372 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0b660-e9b8-45a1-a9f7-2848461f5c7f None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:22.476654 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0b660-e9b8-45a1-a9f7-2848461f5c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:22.488506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4fd850-201b-4b95-a69f-edd70e9fa34f None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:22.494091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-1de0b660-e9b8-45a1-a9f7-2848461f5c7f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:22.495308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1de0b660-e9b8-45a1-a9f7-2848461f5c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:22.563392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 25/50] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:22.586619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0b660-e9b8-45a1-a9f7-2848461f5c7f None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:22.591262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4fd850-201b-4b95-a69f-edd70e9fa34f None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:22.623969 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f69cf7-a1d1-4d33-b665-3483b1250139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:22.624537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f69cf7-a1d1-4d33-b665-3483b1250139 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:22.624820 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f69cf7-a1d1-4d33-b665-3483b1250139 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:22.625090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f69cf7-a1d1-4d33-b665-3483b1250139 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:22.626235 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f69cf7-a1d1-4d33-b665-3483b1250139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:22.628182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f69cf7-a1d1-4d33-b665-3483b1250139 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:22.629901 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 26/51] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:22.635157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f69cf7-a1d1-4d33-b665-3483b1250139 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:22.680776 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:22.684418 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0b660-e9b8-45a1-a9f7-2848461f5c7f None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:22.687038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:22.720944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839162f3-b2c6-4515-b0d4-ec80c44e58c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:22.721587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839162f3-b2c6-4515-b0d4-ec80c44e58c7 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:22.721887 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839162f3-b2c6-4515-b0d4-ec80c44e58c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:22.722162 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839162f3-b2c6-4515-b0d4-ec80c44e58c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:22.732187 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839162f3-b2c6-4515-b0d4-ec80c44e58c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:22.735787 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f69cf7-a1d1-4d33-b665-3483b1250139 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:22.737228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-839162f3-b2c6-4515-b0d4-ec80c44e58c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:22.737228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:22.737228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:22.737228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:22.737228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:22.737228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:22.737228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:22.737228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:22.737228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:22.737228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:22.737228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:22.737228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:22.737228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:22.737228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:22.737228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:22.737228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:22.738188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:22.738188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:22.738188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:22.738188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:22.738188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:22.738188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:22.738188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:22.738188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:22.738188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:22.738188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:22.738188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:22.738188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:22.738188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:22.738188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:22.738188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:22.738188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:22.739043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:22.739043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:22.739043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:22.739043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:22.739043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:22.739043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:22.739043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:22.739043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:22.739043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:22.739043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:22.739043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:22.739043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:22.739043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 26/52] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:39:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:39:22.744210 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839162f3-b2c6-4515-b0d4-ec80c44e58c7 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:22.770736 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1199a-a32b-49cc-8a9e-ca7d22267265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:22.771351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1199a-a32b-49cc-8a9e-ca7d22267265 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:22.771637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1199a-a32b-49cc-8a9e-ca7d22267265 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:22.771914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1199a-a32b-49cc-8a9e-ca7d22267265 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:22.772591 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1199a-a32b-49cc-8a9e-ca7d22267265 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:22.773998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1199a-a32b-49cc-8a9e-ca7d22267265 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:22.802504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 27/53] 10.222.0.65 () {60 vars in 1140 bytes} [Wed Aug 3 12:39:22 2022] POST /identity/v3/projects => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:39:22.845295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839162f3-b2c6-4515-b0d4-ec80c44e58c7 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:22.883274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee5db8-2bac-4dfb-adf7-6b62e22f9cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:22.883921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee5db8-2bac-4dfb-adf7-6b62e22f9cfe None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:22.884207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee5db8-2bac-4dfb-adf7-6b62e22f9cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:22.884487 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee5db8-2bac-4dfb-adf7-6b62e22f9cfe None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:22.885234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee5db8-2bac-4dfb-adf7-6b62e22f9cfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:22.885541 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee5db8-2bac-4dfb-adf7-6b62e22f9cfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:22.887010 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee5db8-2bac-4dfb-adf7-6b62e22f9cfe None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:22.899495 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 27/54] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:22.905224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1199a-a32b-49cc-8a9e-ca7d22267265 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:23.005391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:23.009561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1199a-a32b-49cc-8a9e-ca7d22267265 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:23.045085 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31870bc-90a4-4e75-a331-344cd7621930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:23.045766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31870bc-90a4-4e75-a331-344cd7621930 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:23.046050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31870bc-90a4-4e75-a331-344cd7621930 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:23.046323 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31870bc-90a4-4e75-a331-344cd7621930 None admin] PATH_INFO: `/v3/users/f665e42501ed48eda8620c2548069351` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:23.047561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31870bc-90a4-4e75-a331-344cd7621930 None admin] RBAC: Authorizing `identity:get_user(user_id=f665e42501ed48eda8620c2548069351)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:23.049503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31870bc-90a4-4e75-a331-344cd7621930 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:23.051143 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 28/55] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:22 2022] GET /identity/v3/users/f665e42501ed48eda8620c2548069351 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:23.056717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee5db8-2bac-4dfb-adf7-6b62e22f9cfe None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:23.084821 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:23.158616 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee5db8-2bac-4dfb-adf7-6b62e22f9cfe None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:23.191792 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63743d51-8940-411f-a1b4-7ae60e331425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:23.192528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63743d51-8940-411f-a1b4-7ae60e331425 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:23.192809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63743d51-8940-411f-a1b4-7ae60e331425 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:23.193086 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63743d51-8940-411f-a1b4-7ae60e331425 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:23.194239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63743d51-8940-411f-a1b4-7ae60e331425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:23.195758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63743d51-8940-411f-a1b4-7ae60e331425 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:23.197367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 28/56] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:23.203198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31870bc-90a4-4e75-a331-344cd7621930 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:23.303307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31870bc-90a4-4e75-a331-344cd7621930 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:23.335251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ef8f0-c446-44d8-a9f8-704e23eec733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:23.335888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ef8f0-c446-44d8-a9f8-704e23eec733 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:23.336168 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ef8f0-c446-44d8-a9f8-704e23eec733 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:23.336438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ef8f0-c446-44d8-a9f8-704e23eec733 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:23.337224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ef8f0-c446-44d8-a9f8-704e23eec733 None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.domain.id=default, user.id=f665e42501ed48eda8620c2548069351) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:23.337533 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ef8f0-c446-44d8-a9f8-704e23eec733 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:23.339008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ef8f0-c446-44d8-a9f8-704e23eec733 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:23.344705 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 29/57] 10.222.0.65 () {56 vars in 1322 bytes} [Wed Aug 3 12:39:23 2022] GET /identity/v3/role_assignments?user.id=f665e42501ed48eda8620c2548069351&scope.domain.id=default&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:24.796899 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf97141-5ee4-48dc-84b8-305da902ee21 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:24.797420 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf97141-5ee4-48dc-84b8-305da902ee21 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:24.797506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf97141-5ee4-48dc-84b8-305da902ee21 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:24.799418 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 29/58] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:24.804483 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1d004-2124-4062-bfe7-8cad6712d485 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:24.804910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1d004-2124-4062-bfe7-8cad6712d485 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:24.805038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb1d004-2124-4062-bfe7-8cad6712d485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:24.823736 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1fb1d004-2124-4062-bfe7-8cad6712d485 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:24.825227 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1fb1d004-2124-4062-bfe7-8cad6712d485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:24.847964 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:24.890806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 30/59] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:24.914559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf97141-5ee4-48dc-84b8-305da902ee21 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:25.005195 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:25.012700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf97141-5ee4-48dc-84b8-305da902ee21 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:25.045263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d27626-8f52-460a-8c3b-2b4f7a42b9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:25.045889 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d27626-8f52-460a-8c3b-2b4f7a42b9ac None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:25.046173 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d27626-8f52-460a-8c3b-2b4f7a42b9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:25.046448 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d27626-8f52-460a-8c3b-2b4f7a42b9ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:25.047680 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d27626-8f52-460a-8c3b-2b4f7a42b9ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:25.049121 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d27626-8f52-460a-8c3b-2b4f7a42b9ac None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:25.050730 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 30/60] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:25.056100 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1d004-2124-4062-bfe7-8cad6712d485 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:25.151139 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb1d004-2124-4062-bfe7-8cad6712d485 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:25.183719 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b99577-9938-4de2-90e1-75cb1cf8b85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:25.184337 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b99577-9938-4de2-90e1-75cb1cf8b85a None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:25.184620 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b99577-9938-4de2-90e1-75cb1cf8b85a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:25.184895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b99577-9938-4de2-90e1-75cb1cf8b85a None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:25.185849 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b99577-9938-4de2-90e1-75cb1cf8b85a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:25.187317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b99577-9938-4de2-90e1-75cb1cf8b85a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:25.212787 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 31/61] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:39:25 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:39:28.209190 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b391b4-a91c-4942-bdc1-8e47e9b6c5ee None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:28.209683 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b391b4-a91c-4942-bdc1-8e47e9b6c5ee None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:28.209757 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b391b4-a91c-4942-bdc1-8e47e9b6c5ee None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:28.211367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 31/62] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:28.216687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871ed20-34f4-4cb8-90a9-9d5ec6b9b2db None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:28.217100 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871ed20-34f4-4cb8-90a9-9d5ec6b9b2db None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:28.217271 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871ed20-34f4-4cb8-90a9-9d5ec6b9b2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:28.243046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-3871ed20-34f4-4cb8-90a9-9d5ec6b9b2db None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:28.244263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-3871ed20-34f4-4cb8-90a9-9d5ec6b9b2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:28.258023 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:28.342711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 32/63] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:28.377422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b391b4-a91c-4942-bdc1-8e47e9b6c5ee None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:28.484350 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b391b4-a91c-4942-bdc1-8e47e9b6c5ee None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:28.520301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee2f8d-cf33-4316-a135-3897f6920cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:28.521295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee2f8d-cf33-4316-a135-3897f6920cbf None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:28.521798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee2f8d-cf33-4316-a135-3897f6920cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:28.522272 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee2f8d-cf33-4316-a135-3897f6920cbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:28.533943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee2f8d-cf33-4316-a135-3897f6920cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:28.539352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-30ee2f8d-cf33-4316-a135-3897f6920cbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:28.539352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:28.539352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:28.539352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:28.539352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:28.539352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:28.539352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:28.539352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:28.539352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:28.539352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:28.539352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:28.539352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:28.539352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:28.539352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:28.539352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:28.539352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:28.540180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:28.540180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:28.540180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:28.540180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:28.540180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:28.540180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:28.540180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:28.540180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:28.540180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:28.540180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:28.540180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:28.540180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:28.540180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:28.540180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:28.540180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:28.540180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:28.540943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:28.540943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:28.540943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:28.540943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:28.540943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:28.540943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:28.540943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:28.540943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:28.540943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:28.540943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:28.540943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:28.540943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:28.543270 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 32/64] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:28.550638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871ed20-34f4-4cb8-90a9-9d5ec6b9b2db None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:28.583620 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c2401-ba35-4906-bdc2-1d583fa6dd4e None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:28.583915 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c2401-ba35-4906-bdc2-1d583fa6dd4e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:28.584178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c2401-ba35-4906-bdc2-1d583fa6dd4e None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:28.585798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 33/65] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:28.591080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cb288-95cc-45ea-8b6d-52ea6ba074ff None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:28.591602 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cb288-95cc-45ea-8b6d-52ea6ba074ff None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:28.592083 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cb288-95cc-45ea-8b6d-52ea6ba074ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:28.612919 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-5f1cb288-95cc-45ea-8b6d-52ea6ba074ff None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:28.614530 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-5f1cb288-95cc-45ea-8b6d-52ea6ba074ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:28.639030 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:28.694677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 34/66] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:39:28.719465 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1cb288-95cc-45ea-8b6d-52ea6ba074ff None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:28.741867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871ed20-34f4-4cb8-90a9-9d5ec6b9b2db None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:28.793200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778a632-2610-489d-b6bf-baf263a95484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:28.793839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778a632-2610-489d-b6bf-baf263a95484 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:28.794123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778a632-2610-489d-b6bf-baf263a95484 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:28.794398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778a632-2610-489d-b6bf-baf263a95484 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:28.795176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778a632-2610-489d-b6bf-baf263a95484 None admin] RBAC: Adding query filter params (name=member) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:28.795485 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778a632-2610-489d-b6bf-baf263a95484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:28.796871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3778a632-2610-489d-b6bf-baf263a95484 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:28.806045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 33/67] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:28.810854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffc886-0a99-4953-9ad4-d54933e2d1ec None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:28.810854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffc886-0a99-4953-9ad4-d54933e2d1ec None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:28.810854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ffc886-0a99-4953-9ad4-d54933e2d1ec None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:28.811120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 34/68] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:28.812505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922cb9e-743c-4410-b9e5-91534eee33ba None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:28.812790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922cb9e-743c-4410-b9e5-91534eee33ba None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:28.813066 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922cb9e-743c-4410-b9e5-91534eee33ba None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:28.816112 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 35/69] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:28.816567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922cb9e-743c-4410-b9e5-91534eee33ba None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:28.829920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:28.833945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1cb288-95cc-45ea-8b6d-52ea6ba074ff None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:28.868708 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22484108-b6d4-4c40-a6f7-cb2eb51cbcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:28.869348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22484108-b6d4-4c40-a6f7-cb2eb51cbcfd None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:28.869860 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22484108-b6d4-4c40-a6f7-cb2eb51cbcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:28.869966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22484108-b6d4-4c40-a6f7-cb2eb51cbcfd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:28.880108 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22484108-b6d4-4c40-a6f7-cb2eb51cbcfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:28.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-22484108-b6d4-4c40-a6f7-cb2eb51cbcfd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:39:28.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:28.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:28.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:28.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:28.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:28.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:28.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:28.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:28.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:28.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:28.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:28.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:28.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:28.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:28.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:28.885817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:28.885817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:28.885817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:28.885817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:28.885817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:28.885817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:28.885817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:28.885817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:28.885817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:28.885817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:28.885817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:28.885817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:28.885817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:28.885817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:28.885817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:28.885817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:28.886601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:28.886601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:28.886601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:28.886601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:28.886601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:28.886601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:28.886601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:28.886601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:28.886601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:28.886601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:28.886601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:39:28.886601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:28.886601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 35/70] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:39:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:39:28.887242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5abfe-66e1-4daf-8b1e-130dabb9bf59 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:28.887507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5abfe-66e1-4daf-8b1e-130dabb9bf59 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:28.887778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5abfe-66e1-4daf-8b1e-130dabb9bf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:28.905648 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-56f5abfe-66e1-4daf-8b1e-130dabb9bf59 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:28.906855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-56f5abfe-66e1-4daf-8b1e-130dabb9bf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:28.914559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:28.931089 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922cb9e-743c-4410-b9e5-91534eee33ba None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:28.964269 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f611aa0-872f-4d4f-b6e1-1e6192fb62a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:28.965006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f611aa0-872f-4d4f-b6e1-1e6192fb62a5 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:28.965293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f611aa0-872f-4d4f-b6e1-1e6192fb62a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:28.965564 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f611aa0-872f-4d4f-b6e1-1e6192fb62a5 None admin] PATH_INFO: `/v3/users/75e20284884f42e58636018d16cc77a4` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:28.973482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 36/71] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:28.974731 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a677f5-5c33-44ff-b7fd-bfe03697d203 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:28.975018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a677f5-5c33-44ff-b7fd-bfe03697d203 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:28.975290 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a677f5-5c33-44ff-b7fd-bfe03697d203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:28.982307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f611aa0-872f-4d4f-b6e1-1e6192fb62a5 None admin] RBAC: Authorizing `identity:get_user(user_id=75e20284884f42e58636018d16cc77a4)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:28.983782 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f611aa0-872f-4d4f-b6e1-1e6192fb62a5 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:28.989865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 36/72] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:28 2022] GET /identity/v3/users/75e20284884f42e58636018d16cc77a4 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:28.989865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157f0f0-24d9-470e-acf8-8625206bbd85 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:28.989865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157f0f0-24d9-470e-acf8-8625206bbd85 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:28.989865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157f0f0-24d9-470e-acf8-8625206bbd85 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:28.989865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 37/73] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:28.989865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157f0f0-24d9-470e-acf8-8625206bbd85 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:28.998141 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a0a677f5-5c33-44ff-b7fd-bfe03697d203 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:28.999290 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a0a677f5-5c33-44ff-b7fd-bfe03697d203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:29.054293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:29.065539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 37/74] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:29.066452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a677f5-5c33-44ff-b7fd-bfe03697d203 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:29.091402 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157f0f0-24d9-470e-acf8-8625206bbd85 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:29.124671 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29d98b-b442-4bae-ba13-928362604b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:29.125646 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29d98b-b442-4bae-ba13-928362604b55 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:29.125931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29d98b-b442-4bae-ba13-928362604b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:29.126202 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29d98b-b442-4bae-ba13-928362604b55 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:29.126976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29d98b-b442-4bae-ba13-928362604b55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:29.127283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29d98b-b442-4bae-ba13-928362604b55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:29.128650 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29d98b-b442-4bae-ba13-928362604b55 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:29.136483 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 38/75] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:39:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:29.137645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461fcc5-f20a-43ac-a454-2d3c66157cda None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:29.137928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461fcc5-f20a-43ac-a454-2d3c66157cda None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:29.138205 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461fcc5-f20a-43ac-a454-2d3c66157cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:29.140707 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:29.156763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-9461fcc5-f20a-43ac-a454-2d3c66157cda None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:29.158626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-9461fcc5-f20a-43ac-a454-2d3c66157cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:29.171274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a677f5-5c33-44ff-b7fd-bfe03697d203 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:29.204959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9a47b-8661-44ee-be9f-ff3bd488dd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:29.205533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9a47b-8661-44ee-be9f-ff3bd488dd1b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:29.205811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9a47b-8661-44ee-be9f-ff3bd488dd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:29.206081 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9a47b-8661-44ee-be9f-ff3bd488dd1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:29.225065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 39/76] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:29.225795 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461fcc5-f20a-43ac-a454-2d3c66157cda None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:29.271613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9a47b-8661-44ee-be9f-ff3bd488dd1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:29.273004 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9a47b-8661-44ee-be9f-ff3bd488dd1b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:29.290370 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:29.332279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461fcc5-f20a-43ac-a454-2d3c66157cda None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:29.339508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:29.364915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0d557-04f1-4a2e-923b-02fe61ed8dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:29.365535 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0d557-04f1-4a2e-923b-02fe61ed8dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:29.365815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0d557-04f1-4a2e-923b-02fe61ed8dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:29.366088 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0d557-04f1-4a2e-923b-02fe61ed8dd4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:29.367259 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0d557-04f1-4a2e-923b-02fe61ed8dd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:29.368693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0d557-04f1-4a2e-923b-02fe61ed8dd4 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:29.370270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 40/77] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:29.371051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0d557-04f1-4a2e-923b-02fe61ed8dd4 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:29.371170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 38/78] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:39:29.371918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9a47b-8661-44ee-be9f-ff3bd488dd1b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:29.471522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0d557-04f1-4a2e-923b-02fe61ed8dd4 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:29.471824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9a47b-8661-44ee-be9f-ff3bd488dd1b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:29.495485 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:29.505094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290b0ae-e0e8-40b9-b199-15860ae074aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:29.505532 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd48a5-baa9-4128-9df3-a737dcfb7640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:29.505737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290b0ae-e0e8-40b9-b199-15860ae074aa None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:29.506023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290b0ae-e0e8-40b9-b199-15860ae074aa None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:29.506215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd48a5-baa9-4128-9df3-a737dcfb7640 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:29.506302 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290b0ae-e0e8-40b9-b199-15860ae074aa None admin] PATH_INFO: `/v3/users/75e20284884f42e58636018d16cc77a4` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:29.506534 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd48a5-baa9-4128-9df3-a737dcfb7640 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:29.506808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabd48a5-baa9-4128-9df3-a737dcfb7640 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:29.507547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d290b0ae-e0e8-40b9-b199-15860ae074aa None admin] RBAC: Authorizing `identity:get_user(user_id=75e20284884f42e58636018d16cc77a4)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:29.508975 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d290b0ae-e0e8-40b9-b199-15860ae074aa None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:29.510760 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 39/79] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:29 2022] GET /identity/v3/users/75e20284884f42e58636018d16cc77a4 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:29.511440 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290b0ae-e0e8-40b9-b199-15860ae074aa None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:29.517647 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabd48a5-baa9-4128-9df3-a737dcfb7640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:29.523094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-eabd48a5-baa9-4128-9df3-a737dcfb7640 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:29.523094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:29.523094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:29.523094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:29.523094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:29.523094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:29.523094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:29.523094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:29.523094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:29.523094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:29.523094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:29.523094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:29.523094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:29.523094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:29.523094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:29.523094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:29.523930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:29.523930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:29.523930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:29.523930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:29.523930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:29.523930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:29.523930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:29.523930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:29.523930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:29.523930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:29.523930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:29.523930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:29.523930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:29.523930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:29.523930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:29.523930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:29.524702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:29.524702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:29.524702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:29.524702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:29.524702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:29.524702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:29.524702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:29.524702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:29.524702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:29.524702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:29.524702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:29.524702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:39:29.524702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 41/80] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:39:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:29.525343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd48a5-baa9-4128-9df3-a737dcfb7640 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:29.555446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:29.613022 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290b0ae-e0e8-40b9-b199-15860ae074aa None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:29.621825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabd48a5-baa9-4128-9df3-a737dcfb7640 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:29.645891 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d38a6-1e67-4d70-a85f-de0226192c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:29.646549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d38a6-1e67-4d70-a85f-de0226192c95 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:29.646830 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d38a6-1e67-4d70-a85f-de0226192c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:29.647103 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d38a6-1e67-4d70-a85f-de0226192c95 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:29.655773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6e1da-f51a-4ccd-ae53-821a97b48464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:29.656370 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6e1da-f51a-4ccd-ae53-821a97b48464 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:29.656651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6e1da-f51a-4ccd-ae53-821a97b48464 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:29.656919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6e1da-f51a-4ccd-ae53-821a97b48464 None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:29.657202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d38a6-1e67-4d70-a85f-de0226192c95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:29.657620 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6e1da-f51a-4ccd-ae53-821a97b48464 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:29.659088 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6e1da-f51a-4ccd-ae53-821a97b48464 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:29.662143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-176d38a6-1e67-4d70-a85f-de0226192c95 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:29.662143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:29.662143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:29.662143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:29.662143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:29.662143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:29.662143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:29.662143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:29.662143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:29.662143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:29.662143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:29.662143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:29.662143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:29.662143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:29.662143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:29.662143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:29.663006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:29.663006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:29.663006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:29.663006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:29.663006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:29.663006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:29.663006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:29.663006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:29.663006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:29.663006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:29.663006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:29.663006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:29.663006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:29.663006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:29.663006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:29.663006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:29.668907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:29.668907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:29.668907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:29.668907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:29.668907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:29.668907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:29.668907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:29.668907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:29.668907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:29.668907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:29.668907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:29.668907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:29.668907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 40/81] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:29.668907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d38a6-1e67-4d70-a85f-de0226192c95 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:29.680313 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 42/82] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:39:29 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:39:29.687614 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6e1da-f51a-4ccd-ae53-821a97b48464 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:29.738925 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:29.780429 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:29.783972 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d38a6-1e67-4d70-a85f-de0226192c95 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:29.803841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6e1da-f51a-4ccd-ae53-821a97b48464 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:29.824584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34e90a-ef83-4d1a-a412-defa45d14ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:29.825565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34e90a-ef83-4d1a-a412-defa45d14ece None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:29.825565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34e90a-ef83-4d1a-a412-defa45d14ece None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:29.825808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34e90a-ef83-4d1a-a412-defa45d14ece None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:29.827035 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34e90a-ef83-4d1a-a412-defa45d14ece None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:29.828501 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34e90a-ef83-4d1a-a412-defa45d14ece None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:29.830428 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 41/83] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:29 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:29.831161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34e90a-ef83-4d1a-a412-defa45d14ece None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:29.850775 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c854e6-51cc-4c28-bb85-136c4ad1a4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:29.852069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c854e6-51cc-4c28-bb85-136c4ad1a4ea None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:29.852069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c854e6-51cc-4c28-bb85-136c4ad1a4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:29.852069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c854e6-51cc-4c28-bb85-136c4ad1a4ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:29.935343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c854e6-51cc-4c28-bb85-136c4ad1a4ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:29.936802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c854e6-51cc-4c28-bb85-136c4ad1a4ea None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:29.955016 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34e90a-ef83-4d1a-a412-defa45d14ece None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:29.985530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:29.997181 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c0e08-242c-4022-b72a-079f81b666a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:29.998077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c0e08-242c-4022-b72a-079f81b666a9 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:29.998564 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c0e08-242c-4022-b72a-079f81b666a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:29.999472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c0e08-242c-4022-b72a-079f81b666a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:30.000227 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c0e08-242c-4022-b72a-079f81b666a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:30.000700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c0e08-242c-4022-b72a-079f81b666a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:30.002122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c0e08-242c-4022-b72a-079f81b666a9 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:30.013482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 42/84] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:30.016818 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c0e08-242c-4022-b72a-079f81b666a9 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:30.058920 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:30.060161 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 43/85] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:39:30.061017 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c854e6-51cc-4c28-bb85-136c4ad1a4ea None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:30.146900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c0e08-242c-4022-b72a-079f81b666a9 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:30.188842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c854e6-51cc-4c28-bb85-136c4ad1a4ea None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:30.194430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fce862-2382-4032-9cfb-5bc3e1e2f06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:30.195359 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fce862-2382-4032-9cfb-5bc3e1e2f06b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:30.195901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fce862-2382-4032-9cfb-5bc3e1e2f06b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:30.196376 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fce862-2382-4032-9cfb-5bc3e1e2f06b None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:30.197873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fce862-2382-4032-9cfb-5bc3e1e2f06b None admin] RBAC: Adding query filter params (name=member) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:30.198403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fce862-2382-4032-9cfb-5bc3e1e2f06b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:30.200924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fce862-2382-4032-9cfb-5bc3e1e2f06b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:30.212351 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 43/86] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:30.213767 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fce862-2382-4032-9cfb-5bc3e1e2f06b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:30.227846 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234ef4c-829f-41b8-b092-9e635a011502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:30.228485 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234ef4c-829f-41b8-b092-9e635a011502 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:30.228750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234ef4c-829f-41b8-b092-9e635a011502 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:30.229079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234ef4c-829f-41b8-b092-9e635a011502 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:30.229913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234ef4c-829f-41b8-b092-9e635a011502 None admin] RBAC: Adding query filter params (role.id=0f74302099d9402ba2a922f36cff56ef, scope.project.id=22d0a79f59184a36a955aa3a842c3119, user.id=75e20284884f42e58636018d16cc77a4) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:30.230230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234ef4c-829f-41b8-b092-9e635a011502 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:30.232150 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234ef4c-829f-41b8-b092-9e635a011502 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:30.240108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 44/87] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:30 2022] GET /identity/v3/role_assignments?user.id=75e20284884f42e58636018d16cc77a4&scope.project.id=22d0a79f59184a36a955aa3a842c3119&role.id=0f74302099d9402ba2a922f36cff56ef => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:30.246102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234ef4c-829f-41b8-b092-9e635a011502 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:30.259664 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:30.343902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:30.348349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fce862-2382-4032-9cfb-5bc3e1e2f06b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:30.400705 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860cc64-279f-40d8-b1ff-876399d74dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:30.402863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860cc64-279f-40d8-b1ff-876399d74dcd None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:30.403389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860cc64-279f-40d8-b1ff-876399d74dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:30.405953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860cc64-279f-40d8-b1ff-876399d74dcd None admin] PATH_INFO: `/v3/users/f665e42501ed48eda8620c2548069351` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:30.410770 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860cc64-279f-40d8-b1ff-876399d74dcd None admin] RBAC: Authorizing `identity:get_user(user_id=f665e42501ed48eda8620c2548069351)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:30.412506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860cc64-279f-40d8-b1ff-876399d74dcd None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:30.414999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 44/88] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:30 2022] GET /identity/v3/users/f665e42501ed48eda8620c2548069351 => generated 247 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:30.415850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860cc64-279f-40d8-b1ff-876399d74dcd None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:30.484097 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234ef4c-829f-41b8-b092-9e635a011502 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:30.528442 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860cc64-279f-40d8-b1ff-876399d74dcd None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:30.566008 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05c25b-dcc8-4d0a-b6c5-865fa2247741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:30.566712 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05c25b-dcc8-4d0a-b6c5-865fa2247741 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:30.567049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05c25b-dcc8-4d0a-b6c5-865fa2247741 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:30.567281 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05c25b-dcc8-4d0a-b6c5-865fa2247741 None admin] PATH_INFO: `/v3/users/75e20284884f42e58636018d16cc77a4` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:30.568589 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b05c25b-dcc8-4d0a-b6c5-865fa2247741 None admin] RBAC: Authorizing `identity:get_user(user_id=75e20284884f42e58636018d16cc77a4)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:30.570044 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b05c25b-dcc8-4d0a-b6c5-865fa2247741 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:30.572038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 45/89] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:30 2022] GET /identity/v3/users/75e20284884f42e58636018d16cc77a4 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:30.572811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05c25b-dcc8-4d0a-b6c5-865fa2247741 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:30.580322 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ca43a-bdeb-4986-a3c3-75f240401772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:30.580984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ca43a-bdeb-4986-a3c3-75f240401772 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:30.581264 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ca43a-bdeb-4986-a3c3-75f240401772 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:30.581540 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ca43a-bdeb-4986-a3c3-75f240401772 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:30.587259 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ca43a-bdeb-4986-a3c3-75f240401772 None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:30.588741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ca43a-bdeb-4986-a3c3-75f240401772 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:30.590657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 45/90] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:30 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:30.591265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ca43a-bdeb-4986-a3c3-75f240401772 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:30.607014 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:30.690508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b05c25b-dcc8-4d0a-b6c5-865fa2247741 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:30.727447 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a2692-6bc8-4199-bfd0-42588a339021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:30.728888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a2692-6bc8-4199-bfd0-42588a339021 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:30.729178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a2692-6bc8-4199-bfd0-42588a339021 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:30.729460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a2692-6bc8-4199-bfd0-42588a339021 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:30.760211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:30.812310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ca43a-bdeb-4986-a3c3-75f240401772 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:30.859897 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a2692-6bc8-4199-bfd0-42588a339021 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:30.861372 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a2692-6bc8-4199-bfd0-42588a339021 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:30.880271 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7a0ce-b560-4ecc-aab0-b2bab36a9a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:30.880910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7a0ce-b560-4ecc-aab0-b2bab36a9a86 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:30.881195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7a0ce-b560-4ecc-aab0-b2bab36a9a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:30.881475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7a0ce-b560-4ecc-aab0-b2bab36a9a86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:30.922853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:31.007560 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 46/91] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:39:31.008426 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a2692-6bc8-4199-bfd0-42588a339021 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:31.043355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece7a0ce-b560-4ecc-aab0-b2bab36a9a86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:31.045117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece7a0ce-b560-4ecc-aab0-b2bab36a9a86 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:31.122640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a2692-6bc8-4199-bfd0-42588a339021 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:31.156057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:31.167545 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb58b2-8418-4eb4-90c3-30f4e2b65407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:31.168344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb58b2-8418-4eb4-90c3-30f4e2b65407 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:31.170699 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb58b2-8418-4eb4-90c3-30f4e2b65407 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:31.171187 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb58b2-8418-4eb4-90c3-30f4e2b65407 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:31.172520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb58b2-8418-4eb4-90c3-30f4e2b65407 None admin] RBAC: Adding query filter params (role.id=79ff845a41e241229433560e2b0d6fe5, scope.project.id=22d0a79f59184a36a955aa3a842c3119, user.id=75e20284884f42e58636018d16cc77a4) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:31.174031 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb58b2-8418-4eb4-90c3-30f4e2b65407 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:31.178887 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb58b2-8418-4eb4-90c3-30f4e2b65407 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:31.197562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 47/92] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:31 2022] GET /identity/v3/role_assignments?user.id=75e20284884f42e58636018d16cc77a4&scope.project.id=22d0a79f59184a36a955aa3a842c3119&role.id=79ff845a41e241229433560e2b0d6fe5 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:31.198408 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb58b2-8418-4eb4-90c3-30f4e2b65407 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:31.224414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:31.257633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 46/93] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:39:31.263827 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7a0ce-b560-4ecc-aab0-b2bab36a9a86 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:31.404091 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7a0ce-b560-4ecc-aab0-b2bab36a9a86 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:31.445033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd099b4-c0a5-4ced-ab6c-c99f492047b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:31.446333 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd099b4-c0a5-4ced-ab6c-c99f492047b9 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:31.446892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd099b4-c0a5-4ced-ab6c-c99f492047b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:31.447485 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd099b4-c0a5-4ced-ab6c-c99f492047b9 None admin] PATH_INFO: `/v3/projects/28eb7ee742aa442c8b9db488ae0869b2` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:31.448928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd099b4-c0a5-4ced-ab6c-c99f492047b9 None admin] RBAC: Authorizing `identity:get_project(project_id=28eb7ee742aa442c8b9db488ae0869b2)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:31.450710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd099b4-c0a5-4ced-ab6c-c99f492047b9 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:31.452677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 47/94] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:31 2022] GET /identity/v3/projects/28eb7ee742aa442c8b9db488ae0869b2 => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:31.459486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd099b4-c0a5-4ced-ab6c-c99f492047b9 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:31.514225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:31.576672 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb58b2-8418-4eb4-90c3-30f4e2b65407 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:31.585076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd099b4-c0a5-4ced-ab6c-c99f492047b9 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:31.624237 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad370f-246f-4baa-be12-33109a248ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:31.625207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad370f-246f-4baa-be12-33109a248ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:31.625693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad370f-246f-4baa-be12-33109a248ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:31.626218 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad370f-246f-4baa-be12-33109a248ad5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:31.627444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad370f-246f-4baa-be12-33109a248ad5 None admin] RBAC: Adding query filter params (role.id=0f74302099d9402ba2a922f36cff56ef, scope.project.id=28eb7ee742aa442c8b9db488ae0869b2, user.id=75e20284884f42e58636018d16cc77a4) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:31.628012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad370f-246f-4baa-be12-33109a248ad5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:31.629833 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad370f-246f-4baa-be12-33109a248ad5 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:31.636350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 48/95] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:31 2022] GET /identity/v3/role_assignments?user.id=75e20284884f42e58636018d16cc77a4&scope.project.id=28eb7ee742aa442c8b9db488ae0869b2&role.id=0f74302099d9402ba2a922f36cff56ef => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:31.659972 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a31ee32-bad4-4ae8-84ce-ca8b04059f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:31.660562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31ee32-bad4-4ae8-84ce-ca8b04059f87 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:31.660846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31ee32-bad4-4ae8-84ce-ca8b04059f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:31.661122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a31ee32-bad4-4ae8-84ce-ca8b04059f87 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:31.667689 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31ee32-bad4-4ae8-84ce-ca8b04059f87 None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:31.669210 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a31ee32-bad4-4ae8-84ce-ca8b04059f87 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:31.671422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 48/96] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:31 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 473 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:31.677551 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad370f-246f-4baa-be12-33109a248ad5 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:31.817305 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad370f-246f-4baa-be12-33109a248ad5 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:31.825408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:31.857150 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c3df0b-79b9-40f7-88b5-6e5d47a5c337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:31.858142 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3df0b-79b9-40f7-88b5-6e5d47a5c337 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:31.858747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3df0b-79b9-40f7-88b5-6e5d47a5c337 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:31.859232 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c3df0b-79b9-40f7-88b5-6e5d47a5c337 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:31.860293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3df0b-79b9-40f7-88b5-6e5d47a5c337 None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=22d0a79f59184a36a955aa3a842c3119, user.id=f665e42501ed48eda8620c2548069351) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:31.860844 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3df0b-79b9-40f7-88b5-6e5d47a5c337 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:31.862516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c3df0b-79b9-40f7-88b5-6e5d47a5c337 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:31.869086 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 49/97] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:31 2022] GET /identity/v3/role_assignments?user.id=f665e42501ed48eda8620c2548069351&scope.project.id=22d0a79f59184a36a955aa3a842c3119&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:32.061880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fe62b-07e5-44bb-91b8-9e11ab1a618a None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:32.062247 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fe62b-07e5-44bb-91b8-9e11ab1a618a None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:32.062442 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799fe62b-07e5-44bb-91b8-9e11ab1a618a None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:32.064038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 49/98] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:32.069717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8e9eb-caec-4b08-9ecd-6aba21555785 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:32.070322 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8e9eb-caec-4b08-9ecd-6aba21555785 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:32.070824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d8e9eb-caec-4b08-9ecd-6aba21555785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:32.097205 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-94d8e9eb-caec-4b08-9ecd-6aba21555785 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:32.098837 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-94d8e9eb-caec-4b08-9ecd-6aba21555785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:32.183727 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 50/99] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:32.218750 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799fe62b-07e5-44bb-91b8-9e11ab1a618a None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:32.295007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:32.343255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799fe62b-07e5-44bb-91b8-9e11ab1a618a None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:32.382054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7852ec-c6c3-4700-9252-dfb099346274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:32.382728 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7852ec-c6c3-4700-9252-dfb099346274 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:32.383011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7852ec-c6c3-4700-9252-dfb099346274 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:32.383286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7852ec-c6c3-4700-9252-dfb099346274 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:32.384437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7852ec-c6c3-4700-9252-dfb099346274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:32.385874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7852ec-c6c3-4700-9252-dfb099346274 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:32.387500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 50/100] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:32.393652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8e9eb-caec-4b08-9ecd-6aba21555785 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:32.510466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:32.514115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d8e9eb-caec-4b08-9ecd-6aba21555785 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:32.559734 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee47011-d29e-4ee5-9d6f-a220293e9291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:32.560608 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee47011-d29e-4ee5-9d6f-a220293e9291 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:32.560892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee47011-d29e-4ee5-9d6f-a220293e9291 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:32.561169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee47011-d29e-4ee5-9d6f-a220293e9291 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:32.561888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee47011-d29e-4ee5-9d6f-a220293e9291 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:32.563333 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee47011-d29e-4ee5-9d6f-a220293e9291 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:32.605421 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 51/101] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 3 12:39:32 2022] POST /identity/v3/projects => generated 302 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:39:32.698917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a482706-e120-4553-a438-7efa375549a5 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:32.699445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a482706-e120-4553-a438-7efa375549a5 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:32.699445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a482706-e120-4553-a438-7efa375549a5 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:32.701014 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 51/102] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:32.706928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca53c4-91f2-4d07-b95b-594033cfafca None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:32.707218 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca53c4-91f2-4d07-b95b-594033cfafca None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:32.707500 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca53c4-91f2-4d07-b95b-594033cfafca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:32.732911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-90ca53c4-91f2-4d07-b95b-594033cfafca None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:32.734205 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-90ca53c4-91f2-4d07-b95b-594033cfafca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:32.829636 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 52/103] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:32.858771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a482706-e120-4553-a438-7efa375549a5 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:32.966679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a482706-e120-4553-a438-7efa375549a5 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:32.989411 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16a379-d921-4e49-a5b0-e8705885aee5 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:32.989729 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16a379-d921-4e49-a5b0-e8705885aee5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:32.989969 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16a379-d921-4e49-a5b0-e8705885aee5 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:32.991878 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 53/104] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:32.997449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767cee2-c5f4-447b-bde5-cdc06b3e461c None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:32.997847 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767cee2-c5f4-447b-bde5-cdc06b3e461c None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:32.998222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767cee2-c5f4-447b-bde5-cdc06b3e461c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:33.003293 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:33.010525 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf01161-0a71-4630-b822-4ea104f96b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:33.010525 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf01161-0a71-4630-b822-4ea104f96b4d None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:33.010525 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf01161-0a71-4630-b822-4ea104f96b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:33.010525 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf01161-0a71-4630-b822-4ea104f96b4d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:33.027904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf01161-0a71-4630-b822-4ea104f96b4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:33.028610 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-c767cee2-c5f4-447b-bde5-cdc06b3e461c None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:33.029998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c767cee2-c5f4-447b-bde5-cdc06b3e461c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:33.038549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-aaf01161-0a71-4630-b822-4ea104f96b4d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:39:33.038549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:33.038549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:33.038549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:33.038549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:33.038549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:33.038549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:33.038549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:33.038549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:33.038549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:33.038549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:33.038549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:33.038549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:33.038549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:33.038549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:33.038549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:33.039467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:33.039467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:33.039467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:33.039467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:33.039467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:33.039467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:33.039467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:33.039467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:33.039467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:33.039467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:33.039467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:33.039467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:33.039467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:33.039467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:33.039467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:33.039467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:33.040238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:33.040238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:33.040238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:33.040238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:33.040238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:33.040238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:33.040238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:33.040238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:33.040238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:33.040238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:33.040238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:39:33.040238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:33.040238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 52/105] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:33.042873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf01161-0a71-4630-b822-4ea104f96b4d None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:33.181056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 54/106] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:33.211619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf01161-0a71-4630-b822-4ea104f96b4d None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:33.219739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767cee2-c5f4-447b-bde5-cdc06b3e461c None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:33.249318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:33.280186 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbe4ff-d1d3-434a-85f6-17d3deadf9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:33.280954 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbe4ff-d1d3-434a-85f6-17d3deadf9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:33.281242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbe4ff-d1d3-434a-85f6-17d3deadf9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:33.281546 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbe4ff-d1d3-434a-85f6-17d3deadf9b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:33.282255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbe4ff-d1d3-434a-85f6-17d3deadf9b5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:33.283184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbe4ff-d1d3-434a-85f6-17d3deadf9b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:33.284961 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbe4ff-d1d3-434a-85f6-17d3deadf9b5 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:33.301372 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 53/107] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:33.307758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbe4ff-d1d3-434a-85f6-17d3deadf9b5 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:33.401943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:33.471359 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbe4ff-d1d3-434a-85f6-17d3deadf9b5 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:33.490874 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767cee2-c5f4-447b-bde5-cdc06b3e461c None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:33.505554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65119a40-02aa-44c5-a95b-c32cad53fc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:33.506222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65119a40-02aa-44c5-a95b-c32cad53fc9e None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:33.506530 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65119a40-02aa-44c5-a95b-c32cad53fc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:33.506822 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65119a40-02aa-44c5-a95b-c32cad53fc9e None admin] PATH_INFO: `/v3/users/47ca8e21943b4d2bb3a3fea8353a5bbe` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:33.521463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65119a40-02aa-44c5-a95b-c32cad53fc9e None admin] RBAC: Authorizing `identity:get_user(user_id=47ca8e21943b4d2bb3a3fea8353a5bbe)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:33.523021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65119a40-02aa-44c5-a95b-c32cad53fc9e None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:33.525105 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 54/108] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:33 2022] GET /identity/v3/users/47ca8e21943b4d2bb3a3fea8353a5bbe => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:33.534512 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65119a40-02aa-44c5-a95b-c32cad53fc9e None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:33.548150 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55accff2-6fe8-44da-8965-81d0c7d12a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:33.549003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55accff2-6fe8-44da-8965-81d0c7d12a6f None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:33.549341 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55accff2-6fe8-44da-8965-81d0c7d12a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:33.549618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55accff2-6fe8-44da-8965-81d0c7d12a6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:33.562554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55accff2-6fe8-44da-8965-81d0c7d12a6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:33.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-55accff2-6fe8-44da-8965-81d0c7d12a6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:33.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:33.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:33.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:33.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:33.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:33.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:33.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:33.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:33.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:33.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:33.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:33.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:33.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:33.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:33.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:33.570704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:33.570704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:33.570704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:33.570704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:33.570704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:33.570704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:33.570704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:33.570704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:33.570704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:33.570704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:33.570704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:33.570704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:33.570704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:33.570704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:33.570704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:33.570704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:33.571504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:33.571504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:33.571504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:33.571504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:33.571504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:33.571504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:33.571504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:33.571504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:33.571504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:33.571504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:33.571504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:33.571504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:39:33.571504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 55/109] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:33.577609 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55accff2-6fe8-44da-8965-81d0c7d12a6f None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:33.666864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:33.696135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55accff2-6fe8-44da-8965-81d0c7d12a6f None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:33.743392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19809f3b-3d90-4b3d-97c9-2fe614a9dadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:33.744125 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19809f3b-3d90-4b3d-97c9-2fe614a9dadd None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:33.744498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19809f3b-3d90-4b3d-97c9-2fe614a9dadd None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:33.744845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19809f3b-3d90-4b3d-97c9-2fe614a9dadd None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:33.745639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19809f3b-3d90-4b3d-97c9-2fe614a9dadd None admin] RBAC: Adding query filter params (name=member) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:33.746015 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19809f3b-3d90-4b3d-97c9-2fe614a9dadd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:33.747674 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19809f3b-3d90-4b3d-97c9-2fe614a9dadd None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:33.754587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65119a40-02aa-44c5-a95b-c32cad53fc9e None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:33.766604 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 56/110] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:33.768077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:33.769602 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19809f3b-3d90-4b3d-97c9-2fe614a9dadd None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:33.793361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a42d5-b910-4406-811f-b31ec73961cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:33.794319 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a42d5-b910-4406-811f-b31ec73961cd None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:33.794883 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a42d5-b910-4406-811f-b31ec73961cd None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:33.795452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a42d5-b910-4406-811f-b31ec73961cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:33.898005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a42d5-b910-4406-811f-b31ec73961cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:33.900366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a42d5-b910-4406-811f-b31ec73961cd None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:33.970462 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19809f3b-3d90-4b3d-97c9-2fe614a9dadd None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:34.040001 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86631c32-1e16-44ff-96d6-2fb72f94fdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:34.040609 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86631c32-1e16-44ff-96d6-2fb72f94fdfb None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:34.040893 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86631c32-1e16-44ff-96d6-2fb72f94fdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:34.041173 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86631c32-1e16-44ff-96d6-2fb72f94fdfb None admin] PATH_INFO: `/v3/users/75e20284884f42e58636018d16cc77a4` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:34.042402 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86631c32-1e16-44ff-96d6-2fb72f94fdfb None admin] RBAC: Authorizing `identity:get_user(user_id=75e20284884f42e58636018d16cc77a4)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:34.043904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86631c32-1e16-44ff-96d6-2fb72f94fdfb None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:34.046066 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 57/111] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:33 2022] GET /identity/v3/users/75e20284884f42e58636018d16cc77a4 => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:34.052848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86631c32-1e16-44ff-96d6-2fb72f94fdfb None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:34.057106 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:34.076627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 55/112] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:39:34.084087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a42d5-b910-4406-811f-b31ec73961cd None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:34.102860 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:34.246754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86631c32-1e16-44ff-96d6-2fb72f94fdfb None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:34.260473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a42d5-b910-4406-811f-b31ec73961cd None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:34.315905 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4342eb3-a6c3-442a-8d27-5da1a2a8db58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:34.316485 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4342eb3-a6c3-442a-8d27-5da1a2a8db58 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:34.316779 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4342eb3-a6c3-442a-8d27-5da1a2a8db58 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:34.317059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4342eb3-a6c3-442a-8d27-5da1a2a8db58 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:34.318223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfdc55-c961-4676-8b97-0057e098c875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:34.318903 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfdc55-c961-4676-8b97-0057e098c875 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:34.319185 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfdc55-c961-4676-8b97-0057e098c875 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:34.319462 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfdc55-c961-4676-8b97-0057e098c875 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:34.320622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4342eb3-a6c3-442a-8d27-5da1a2a8db58 None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:34.322084 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4342eb3-a6c3-442a-8d27-5da1a2a8db58 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:34.324507 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfdc55-c961-4676-8b97-0057e098c875 None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:34.325938 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfdc55-c961-4676-8b97-0057e098c875 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:34.331975 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 58/113] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:34 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:34.332862 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 56/114] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:34 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:34.333346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10ae22-3122-47a3-9bd7-7cce014cda49 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:34.333638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10ae22-3122-47a3-9bd7-7cce014cda49 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:34.333920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10ae22-3122-47a3-9bd7-7cce014cda49 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:34.335178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 59/115] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:34.336990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4342eb3-a6c3-442a-8d27-5da1a2a8db58 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:34.341506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b2d3d-4860-4284-9488-bd76ac859ce8 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:34.341795 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b2d3d-4860-4284-9488-bd76ac859ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:34.342077 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b2d3d-4860-4284-9488-bd76ac859ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:34.377675 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-9a5b2d3d-4860-4284-9488-bd76ac859ce8 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:34.379759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-9a5b2d3d-4860-4284-9488-bd76ac859ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:34.473226 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:34.487773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:34.494513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 60/116] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:34.496648 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b2d3d-4860-4284-9488-bd76ac859ce8 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:34.520155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4342eb3-a6c3-442a-8d27-5da1a2a8db58 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:34.593046 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ea6db-c508-4aaa-be7d-ef05ece766f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:34.594056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ea6db-c508-4aaa-be7d-ef05ece766f0 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:34.594553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ea6db-c508-4aaa-be7d-ef05ece766f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:34.595563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ea6db-c508-4aaa-be7d-ef05ece766f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:34.596551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ea6db-c508-4aaa-be7d-ef05ece766f0 None admin] RBAC: Adding query filter params (role.id=ebfd821c64dd448cae68d5408ae32993, scope.project.id=22d0a79f59184a36a955aa3a842c3119, user.id=47ca8e21943b4d2bb3a3fea8353a5bbe) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:34.597073 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ea6db-c508-4aaa-be7d-ef05ece766f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:34.598819 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ea6db-c508-4aaa-be7d-ef05ece766f0 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:34.605212 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 57/117] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:34 2022] GET /identity/v3/role_assignments?user.id=47ca8e21943b4d2bb3a3fea8353a5bbe&scope.project.id=22d0a79f59184a36a955aa3a842c3119&role.id=ebfd821c64dd448cae68d5408ae32993 => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:34.606235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ea6db-c508-4aaa-be7d-ef05ece766f0 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:34.721263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b2d3d-4860-4284-9488-bd76ac859ce8 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:34.774519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5035a-c1bd-4e6a-8e14-7353fc23fb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:34.775211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5035a-c1bd-4e6a-8e14-7353fc23fb61 None admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:34.775495 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5035a-c1bd-4e6a-8e14-7353fc23fb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:34.775787 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5035a-c1bd-4e6a-8e14-7353fc23fb61 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119/users/75e20284884f42e58636018d16cc77a4/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:34.779059 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5035a-c1bd-4e6a-8e14-7353fc23fb61 None admin] RBAC: Authorizing `identity:create_grant(project_id=22d0a79f59184a36a955aa3a842c3119, user_id=75e20284884f42e58636018d16cc77a4, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:34.780548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5035a-c1bd-4e6a-8e14-7353fc23fb61 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:34.807690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ea6db-c508-4aaa-be7d-ef05ece766f0 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:34.817539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 61/118] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:39:34 2022] PUT /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119/users/75e20284884f42e58636018d16cc77a4/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:39:34.818905 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4a189-e0c7-4e46-8805-6c1ef939a661 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:34.819195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4a189-e0c7-4e46-8805-6c1ef939a661 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:34.819524 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4a189-e0c7-4e46-8805-6c1ef939a661 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:34.820982 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 62/119] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:34.822623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb15fb-2ee7-448a-98e5-fc102c54c019 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:34.822909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb15fb-2ee7-448a-98e5-fc102c54c019 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:34.823185 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb15fb-2ee7-448a-98e5-fc102c54c019 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:34.824865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 63/120] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:34.828204 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d8637-0fc7-4de3-b1f6-7b90299b261c None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:34.828587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d8637-0fc7-4de3-b1f6-7b90299b261c None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:34.828965 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d8637-0fc7-4de3-b1f6-7b90299b261c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:34.852164 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-876d8637-0fc7-4de3-b1f6-7b90299b261c None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:34.853334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-876d8637-0fc7-4de3-b1f6-7b90299b261c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:34.912838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a312f7df-995a-4b0c-ae2c-08f059e9b660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:34.913557 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312f7df-995a-4b0c-ae2c-08f059e9b660 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:34.913840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312f7df-995a-4b0c-ae2c-08f059e9b660 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:34.914115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a312f7df-995a-4b0c-ae2c-08f059e9b660 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:34.933690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:34.938524 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:34.943194 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a312f7df-995a-4b0c-ae2c-08f059e9b660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:34.952658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-a312f7df-995a-4b0c-ae2c-08f059e9b660 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:39:34.952658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:34.952658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:34.952658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:34.952658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:34.952658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:34.952658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:34.952658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:34.952658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:34.952658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:34.952658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:34.952658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:34.952658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:34.952658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:34.952658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:34.952658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:34.953583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:34.953583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:34.953583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:34.953583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:34.953583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:34.953583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:34.953583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:34.953583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:34.953583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:34.953583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:34.953583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:34.953583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:34.953583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:34.953583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:34.953583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:34.953583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:34.954917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:34.954917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:34.954917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:34.954917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:34.954917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:34.954917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:34.954917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:34.954917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:34.954917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:34.954917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:34.954917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:39:34.954917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:34.954917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 58/121] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:39:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:34.956297 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f94b4d-dac2-4a5b-89cf-f832d1b70c4f None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:34.956297 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f94b4d-dac2-4a5b-89cf-f832d1b70c4f None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:34.956548 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f94b4d-dac2-4a5b-89cf-f832d1b70c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:34.960660 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 64/122] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:34.963895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d8637-0fc7-4de3-b1f6-7b90299b261c None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:35.000379 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a7f94b4d-dac2-4a5b-89cf-f832d1b70c4f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:35.001622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a7f94b4d-dac2-4a5b-89cf-f832d1b70c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:35.075120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 59/123] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:35.075969 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f94b4d-dac2-4a5b-89cf-f832d1b70c4f None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:35.130956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d8637-0fc7-4de3-b1f6-7b90299b261c None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:35.182579 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f94b4d-dac2-4a5b-89cf-f832d1b70c4f None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:35.211454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f52cea-7e1d-45a6-a659-70d0f12e1e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:35.212199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f52cea-7e1d-45a6-a659-70d0f12e1e87 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:35.212492 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f52cea-7e1d-45a6-a659-70d0f12e1e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:35.212775 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f52cea-7e1d-45a6-a659-70d0f12e1e87 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:35.213584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f52cea-7e1d-45a6-a659-70d0f12e1e87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:35.213781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:35.213881 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f52cea-7e1d-45a6-a659-70d0f12e1e87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:35.215476 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f52cea-7e1d-45a6-a659-70d0f12e1e87 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:35.225910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1bf8f-a0a8-4649-9470-5648c2e61167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:35.226606 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1bf8f-a0a8-4649-9470-5648c2e61167 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:35.226890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1bf8f-a0a8-4649-9470-5648c2e61167 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:35.227162 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1bf8f-a0a8-4649-9470-5648c2e61167 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:35.232388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 65/124] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:39:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:35.233254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f52cea-7e1d-45a6-a659-70d0f12e1e87 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:35.248775 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1bf8f-a0a8-4649-9470-5648c2e61167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:35.259584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-52e1bf8f-a0a8-4649-9470-5648c2e61167 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:35.259584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:35.259584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:35.259584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:35.259584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:35.259584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:35.259584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:35.259584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:35.259584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:35.259584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:35.259584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:35.259584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:35.259584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:35.259584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:35.259584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:35.259584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:35.260500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:35.260500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:35.260500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:35.260500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:35.260500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:35.260500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:35.260500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:35.260500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:35.260500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:35.260500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:35.260500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:35.260500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:35.260500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:35.260500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:35.260500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:35.260500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:35.261276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:35.261276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:35.261276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:35.261276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:35.261276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:35.261276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:35.261276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:35.261276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:35.261276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:35.261276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:35.261276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:35.261276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:35.261276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 60/125] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:35.262367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1bf8f-a0a8-4649-9470-5648c2e61167 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:35.346868 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:35.420757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f52cea-7e1d-45a6-a659-70d0f12e1e87 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:35.456092 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1bf8f-a0a8-4649-9470-5648c2e61167 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:35.481892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e8e6d-f9d6-4de8-bd6a-5b21b37a2750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:35.482558 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e8e6d-f9d6-4de8-bd6a-5b21b37a2750 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:35.482842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e8e6d-f9d6-4de8-bd6a-5b21b37a2750 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:35.483114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e8e6d-f9d6-4de8-bd6a-5b21b37a2750 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:35.503703 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e8e6d-f9d6-4de8-bd6a-5b21b37a2750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:35.511224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-337e8e6d-f9d6-4de8-bd6a-5b21b37a2750 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:35.511224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:35.511224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:35.511224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:35.511224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:35.511224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:35.511224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:35.511224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:35.511224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:35.511224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:35.511224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:35.511224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:35.511224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:35.511224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:35.511224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:35.511224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:35.512068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:35.512068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:35.512068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:35.512068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:35.512068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:35.512068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:35.512068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:35.512068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:35.512068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:35.512068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:35.512068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:35.512068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:35.512068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:35.512068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:35.512068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:35.512068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:35.512830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:35.512830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:35.512830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:35.512830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:35.512830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:35.512830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:35.512830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:35.512830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:35.512830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:35.512830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:35.512830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:35.512830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:39:35.513854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 66/126] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:39:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:35.517981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e8e6d-f9d6-4de8-bd6a-5b21b37a2750 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:35.520080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16057e0-7706-409b-b37a-2e8a448b59d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:35.520729 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16057e0-7706-409b-b37a-2e8a448b59d5 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:35.521068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16057e0-7706-409b-b37a-2e8a448b59d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:35.521344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16057e0-7706-409b-b37a-2e8a448b59d5 None admin] PATH_INFO: `/v3/users/75e20284884f42e58636018d16cc77a4` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:35.522656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16057e0-7706-409b-b37a-2e8a448b59d5 None admin] RBAC: Authorizing `identity:get_user(user_id=75e20284884f42e58636018d16cc77a4)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:35.525112 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16057e0-7706-409b-b37a-2e8a448b59d5 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:35.526753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 61/127] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:35 2022] GET /identity/v3/users/75e20284884f42e58636018d16cc77a4 => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:35.527507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16057e0-7706-409b-b37a-2e8a448b59d5 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:35.638418 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:35.706322 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e8e6d-f9d6-4de8-bd6a-5b21b37a2750 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:35.711517 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16057e0-7706-409b-b37a-2e8a448b59d5 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:35.731830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:35.767784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932ea8f-e0aa-4f12-a8f6-39661ef7fc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:35.768403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932ea8f-e0aa-4f12-a8f6-39661ef7fc97 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:35.768686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932ea8f-e0aa-4f12-a8f6-39661ef7fc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:35.768958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932ea8f-e0aa-4f12-a8f6-39661ef7fc97 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:35.769674 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932ea8f-e0aa-4f12-a8f6-39661ef7fc97 None admin] RBAC: Adding query filter params (name=member) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:35.769974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932ea8f-e0aa-4f12-a8f6-39661ef7fc97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:35.771440 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932ea8f-e0aa-4f12-a8f6-39661ef7fc97 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:35.775710 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc2219-be6a-4183-9c39-9f95f2a7fffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:35.778755 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc2219-be6a-4183-9c39-9f95f2a7fffa None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:35.779070 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc2219-be6a-4183-9c39-9f95f2a7fffa None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:35.779319 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc2219-be6a-4183-9c39-9f95f2a7fffa None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:35.780046 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc2219-be6a-4183-9c39-9f95f2a7fffa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:35.780347 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc2219-be6a-4183-9c39-9f95f2a7fffa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:35.781698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc2219-be6a-4183-9c39-9f95f2a7fffa None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:35.788269 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 67/128] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:35.789093 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932ea8f-e0aa-4f12-a8f6-39661ef7fc97 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:35.804470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 62/129] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:35.805404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc2219-be6a-4183-9c39-9f95f2a7fffa None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:35.906392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932ea8f-e0aa-4f12-a8f6-39661ef7fc97 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:35.918744 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc2219-be6a-4183-9c39-9f95f2a7fffa None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:35.940921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:35.944647 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e20973-daa7-44d1-a8a4-bd9a3ea38bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:35.945572 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e20973-daa7-44d1-a8a4-bd9a3ea38bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:35.946048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e20973-daa7-44d1-a8a4-bd9a3ea38bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:35.946560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e20973-daa7-44d1-a8a4-bd9a3ea38bf3 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:35.948048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e20973-daa7-44d1-a8a4-bd9a3ea38bf3 None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:35.949698 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e20973-daa7-44d1-a8a4-bd9a3ea38bf3 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:35.951990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 68/130] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:35 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:35.952959 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e20973-daa7-44d1-a8a4-bd9a3ea38bf3 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:35.961066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d72fc-1510-4e98-bf6b-3fa88fdd4988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:35.961724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d72fc-1510-4e98-bf6b-3fa88fdd4988 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:35.962008 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d72fc-1510-4e98-bf6b-3fa88fdd4988 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:35.962286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d72fc-1510-4e98-bf6b-3fa88fdd4988 None admin] PATH_INFO: `/v3/users/75e20284884f42e58636018d16cc77a4` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:35.963539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d72fc-1510-4e98-bf6b-3fa88fdd4988 None admin] RBAC: Authorizing `identity:get_user(user_id=75e20284884f42e58636018d16cc77a4)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:35.965101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d72fc-1510-4e98-bf6b-3fa88fdd4988 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:35.967109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 63/131] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:35 2022] GET /identity/v3/users/75e20284884f42e58636018d16cc77a4 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:35.968400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7f276-fb1f-4598-b40d-c0f16fbd7064 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:35.968891 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7f276-fb1f-4598-b40d-c0f16fbd7064 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:35.969065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab7f276-fb1f-4598-b40d-c0f16fbd7064 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:35.970955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 64/132] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:35.975192 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7f276-fb1f-4598-b40d-c0f16fbd7064 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:36.007256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:36.069228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e20973-daa7-44d1-a8a4-bd9a3ea38bf3 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:36.082515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab7f276-fb1f-4598-b40d-c0f16fbd7064 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:36.106372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05580ef0-d54b-436b-9e8a-61de0883563a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:36.107450 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05580ef0-d54b-436b-9e8a-61de0883563a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:36.107929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05580ef0-d54b-436b-9e8a-61de0883563a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:36.108502 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05580ef0-d54b-436b-9e8a-61de0883563a None admin] PATH_INFO: `/v3/users/f665e42501ed48eda8620c2548069351` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:36.110176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05580ef0-d54b-436b-9e8a-61de0883563a None admin] RBAC: Authorizing `identity:get_user(user_id=f665e42501ed48eda8620c2548069351)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:36.112676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05580ef0-d54b-436b-9e8a-61de0883563a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:36.115126 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 69/133] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:35 2022] GET /identity/v3/users/f665e42501ed48eda8620c2548069351 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:36.116140 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05580ef0-d54b-436b-9e8a-61de0883563a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:36.125836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c586e6ff-45fd-4dab-9a89-191f8f71e63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:36.126920 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586e6ff-45fd-4dab-9a89-191f8f71e63b None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:36.127027 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586e6ff-45fd-4dab-9a89-191f8f71e63b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:36.127244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c586e6ff-45fd-4dab-9a89-191f8f71e63b None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119/users/75e20284884f42e58636018d16cc77a4/roles/79ff845a41e241229433560e2b0d6fe5` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:36.129570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586e6ff-45fd-4dab-9a89-191f8f71e63b None admin] RBAC: Authorizing `identity:create_grant(project_id=22d0a79f59184a36a955aa3a842c3119, user_id=75e20284884f42e58636018d16cc77a4, role_id=79ff845a41e241229433560e2b0d6fe5)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:36.131154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c586e6ff-45fd-4dab-9a89-191f8f71e63b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:36.158617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 65/134] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:39:35 2022] PUT /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119/users/75e20284884f42e58636018d16cc77a4/roles/79ff845a41e241229433560e2b0d6fe5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:39:36.160262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce162e9-c993-42a2-b34e-df24fb3b16a1 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:36.160561 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce162e9-c993-42a2-b34e-df24fb3b16a1 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:36.160843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce162e9-c993-42a2-b34e-df24fb3b16a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:36.206672 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-dce162e9-c993-42a2-b34e-df24fb3b16a1 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:36.207913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-dce162e9-c993-42a2-b34e-df24fb3b16a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:36.241667 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05580ef0-d54b-436b-9e8a-61de0883563a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:36.280715 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a9e1e-f87d-416e-9813-4cf40b489244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:36.281850 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a9e1e-f87d-416e-9813-4cf40b489244 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:36.282392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a9e1e-f87d-416e-9813-4cf40b489244 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:36.283475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a9e1e-f87d-416e-9813-4cf40b489244 None admin] PATH_INFO: `/v3/projects/28eb7ee742aa442c8b9db488ae0869b2` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:36.284925 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a9e1e-f87d-416e-9813-4cf40b489244 None admin] RBAC: Authorizing `identity:get_project(project_id=28eb7ee742aa442c8b9db488ae0869b2)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:36.286982 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a9e1e-f87d-416e-9813-4cf40b489244 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:36.289068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 70/135] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:36 2022] GET /identity/v3/projects/28eb7ee742aa442c8b9db488ae0869b2 => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:36.289583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:36.295267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a9e1e-f87d-416e-9813-4cf40b489244 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:36.310778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:36.351891 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 66/136] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:36.352743 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce162e9-c993-42a2-b34e-df24fb3b16a1 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:36.444299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a9e1e-f87d-416e-9813-4cf40b489244 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:36.484666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf37dd-ed5b-458d-9e87-15d3345e4b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:36.486193 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf37dd-ed5b-458d-9e87-15d3345e4b13 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:36.486921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf37dd-ed5b-458d-9e87-15d3345e4b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:36.487368 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf37dd-ed5b-458d-9e87-15d3345e4b13 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:36.489422 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf37dd-ed5b-458d-9e87-15d3345e4b13 None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:36.491773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf37dd-ed5b-458d-9e87-15d3345e4b13 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:36.494831 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 71/137] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:36 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:36.496444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf37dd-ed5b-458d-9e87-15d3345e4b13 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:36.498113 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce162e9-c993-42a2-b34e-df24fb3b16a1 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:36.535901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e64af-643f-42ac-9790-4321f6a1624c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:36.536734 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e64af-643f-42ac-9790-4321f6a1624c None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:36.537105 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e64af-643f-42ac-9790-4321f6a1624c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:36.537421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e64af-643f-42ac-9790-4321f6a1624c None admin] PATH_INFO: `/v3/projects/28eb7ee742aa442c8b9db488ae0869b2/users/75e20284884f42e58636018d16cc77a4/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:36.539547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e64af-643f-42ac-9790-4321f6a1624c None admin] RBAC: Authorizing `identity:create_grant(project_id=28eb7ee742aa442c8b9db488ae0869b2, user_id=75e20284884f42e58636018d16cc77a4, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:36.541262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e64af-643f-42ac-9790-4321f6a1624c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:36.557944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:36.573493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 67/138] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:39:36 2022] PUT /identity/v3/projects/28eb7ee742aa442c8b9db488ae0869b2/users/75e20284884f42e58636018d16cc77a4/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:39:36.574520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e64af-643f-42ac-9790-4321f6a1624c None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:36.626321 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf37dd-ed5b-458d-9e87-15d3345e4b13 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:36.655084 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:36.674914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857415eb-0c3f-42a9-aa25-fb8c789df259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:36.676229 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857415eb-0c3f-42a9-aa25-fb8c789df259 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:36.676715 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857415eb-0c3f-42a9-aa25-fb8c789df259 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:36.677650 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857415eb-0c3f-42a9-aa25-fb8c789df259 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:36.679479 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857415eb-0c3f-42a9-aa25-fb8c789df259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:36.681148 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857415eb-0c3f-42a9-aa25-fb8c789df259 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:36.683678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 72/139] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:36.690415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857415eb-0c3f-42a9-aa25-fb8c789df259 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:36.807235 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857415eb-0c3f-42a9-aa25-fb8c789df259 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:36.830330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e64af-643f-42ac-9790-4321f6a1624c None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:36.837519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:36.866281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553294ca-4d10-43a1-85ad-a69be4e132f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:36.866947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553294ca-4d10-43a1-85ad-a69be4e132f0 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:36.867228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553294ca-4d10-43a1-85ad-a69be4e132f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:36.867824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553294ca-4d10-43a1-85ad-a69be4e132f0 None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:36.868544 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553294ca-4d10-43a1-85ad-a69be4e132f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:36.870257 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553294ca-4d10-43a1-85ad-a69be4e132f0 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:36.906032 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef10ca-7418-4b0f-b5eb-7de4cc421123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:36.906826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef10ca-7418-4b0f-b5eb-7de4cc421123 None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:36.907269 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 73/140] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:39:36 2022] POST /identity/v3/users => generated 283 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:39:36.907406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef10ca-7418-4b0f-b5eb-7de4cc421123 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:36.907572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef10ca-7418-4b0f-b5eb-7de4cc421123 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119/users/f665e42501ed48eda8620c2548069351/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:36.908372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943335f7-99b6-4bd1-8abd-7bab3ee36259 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:36.908658 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943335f7-99b6-4bd1-8abd-7bab3ee36259 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:36.908928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943335f7-99b6-4bd1-8abd-7bab3ee36259 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:36.910324 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef10ca-7418-4b0f-b5eb-7de4cc421123 None admin] RBAC: Authorizing `identity:create_grant(project_id=22d0a79f59184a36a955aa3a842c3119, user_id=f665e42501ed48eda8620c2548069351, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:36.910498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 74/141] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:36.912726 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef10ca-7418-4b0f-b5eb-7de4cc421123 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:36.917115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f3ead-21f5-46e8-9f75-1fe91308a6f7 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:36.917414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f3ead-21f5-46e8-9f75-1fe91308a6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:36.917692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f3ead-21f5-46e8-9f75-1fe91308a6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:36.932131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 68/142] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:39:36 2022] PUT /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119/users/f665e42501ed48eda8620c2548069351/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:39:36.938407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-817f3ead-21f5-46e8-9f75-1fe91308a6f7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:36.940269 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-817f3ead-21f5-46e8-9f75-1fe91308a6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:37.020304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 75/143] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:39:37.042627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef10ca-7418-4b0f-b5eb-7de4cc421123 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:37.178058 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:37.183186 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef10ca-7418-4b0f-b5eb-7de4cc421123 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:37.228753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8056f-180f-4d5c-9315-eb6669b10aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:37.229425 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8056f-180f-4d5c-9315-eb6669b10aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:37.229781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8056f-180f-4d5c-9315-eb6669b10aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:37.230062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8056f-180f-4d5c-9315-eb6669b10aa2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:37.242724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8056f-180f-4d5c-9315-eb6669b10aa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:37.247945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-b9c8056f-180f-4d5c-9315-eb6669b10aa2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:39:37.247945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:37.247945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:37.247945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:37.247945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:37.247945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:37.247945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:37.247945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:37.247945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:37.247945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:37.247945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:37.247945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:37.247945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:37.247945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:37.247945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:37.247945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:37.249999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:37.249999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:37.249999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:37.249999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:37.249999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:37.249999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:37.249999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:37.249999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:37.249999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:37.249999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:37.249999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:37.249999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:37.249999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:37.249999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:37.249999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:37.249999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:37.251165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:37.251165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:37.251165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:37.251165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:37.251165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:37.251165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:37.251165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:37.251165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:37.251165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:37.251165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:37.251165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:39:37.251165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:37.251165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 69/144] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:37.255869 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f3ead-21f5-46e8-9f75-1fe91308a6f7 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:37.366925 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:37.371025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f3ead-21f5-46e8-9f75-1fe91308a6f7 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:37.413519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751e5b8-0fd1-41ed-8738-362d40598388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:37.414153 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751e5b8-0fd1-41ed-8738-362d40598388 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:37.414430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751e5b8-0fd1-41ed-8738-362d40598388 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:37.414993 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751e5b8-0fd1-41ed-8738-362d40598388 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:37.415761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751e5b8-0fd1-41ed-8738-362d40598388 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:37.416034 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751e5b8-0fd1-41ed-8738-362d40598388 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:37.417424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751e5b8-0fd1-41ed-8738-362d40598388 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:37.426793 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 76/145] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:37.432812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8056f-180f-4d5c-9315-eb6669b10aa2 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:37.465749 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341f0a9-78a8-4c3b-81e2-4da9a513ccc6 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:37.466040 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341f0a9-78a8-4c3b-81e2-4da9a513ccc6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:37.466304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341f0a9-78a8-4c3b-81e2-4da9a513ccc6 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:37.467834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 77/146] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:37.473164 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31710a94-5b7a-4cab-927b-326c2986a508 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:37.473452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31710a94-5b7a-4cab-927b-326c2986a508 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:37.473728 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31710a94-5b7a-4cab-927b-326c2986a508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:37.494222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-31710a94-5b7a-4cab-927b-326c2986a508 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:37.495419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-31710a94-5b7a-4cab-927b-326c2986a508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:37.582524 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 78/147] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:37.609921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31710a94-5b7a-4cab-927b-326c2986a508 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:37.614861 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8056f-180f-4d5c-9315-eb6669b10aa2 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:37.685338 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7fab5-bad3-44fc-a91f-4af6e836c6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:37.685928 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7fab5-bad3-44fc-a91f-4af6e836c6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:37.686215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7fab5-bad3-44fc-a91f-4af6e836c6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:37.686515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7fab5-bad3-44fc-a91f-4af6e836c6b5 None admin] PATH_INFO: `/v3/users/47ca8e21943b4d2bb3a3fea8353a5bbe` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:37.691341 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7fab5-bad3-44fc-a91f-4af6e836c6b5 None admin] RBAC: Authorizing `identity:get_user(user_id=47ca8e21943b4d2bb3a3fea8353a5bbe)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:37.692853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7fab5-bad3-44fc-a91f-4af6e836c6b5 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:37.694680 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 70/148] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:37 2022] GET /identity/v3/users/47ca8e21943b4d2bb3a3fea8353a5bbe => generated 289 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:37.702313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7fab5-bad3-44fc-a91f-4af6e836c6b5 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:37.728522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:37.743569 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31710a94-5b7a-4cab-927b-326c2986a508 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:37.764287 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:37.797043 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714d2d1-927e-4a20-8785-f8c7b5e5614d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:37.798596 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714d2d1-927e-4a20-8785-f8c7b5e5614d None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:37.799084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714d2d1-927e-4a20-8785-f8c7b5e5614d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:37.799553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714d2d1-927e-4a20-8785-f8c7b5e5614d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:37.813734 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714d2d1-927e-4a20-8785-f8c7b5e5614d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:37.819330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-a714d2d1-927e-4a20-8785-f8c7b5e5614d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:37.819330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:37.819330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:37.819330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:37.819330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:37.819330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:37.819330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:37.819330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:37.819330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:37.819330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:37.819330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:37.819330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:37.819330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:37.819330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:37.819330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:37.819330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:37.820156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:37.820156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:37.820156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:37.820156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:37.820156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:37.820156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:37.820156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:37.820156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:37.820156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:37.820156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:37.820156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:37.820156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:37.820156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:37.820156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:37.820156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:37.820156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:37.820921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:37.820921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:37.820921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:37.820921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:37.820921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:37.820921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:37.820921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:37.820921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:37.820921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:37.820921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:37.820921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:37.820921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:39:37.823507 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 79/149] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:37.830234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714d2d1-927e-4a20-8785-f8c7b5e5614d None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:37.940552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7fab5-bad3-44fc-a91f-4af6e836c6b5 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:37.954877 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714d2d1-927e-4a20-8785-f8c7b5e5614d None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:38.000260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd74f3-26b4-47eb-b333-2f6db0224c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:38.001229 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd74f3-26b4-47eb-b333-2f6db0224c70 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:38.001521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd74f3-26b4-47eb-b333-2f6db0224c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:38.001797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd74f3-26b4-47eb-b333-2f6db0224c70 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:38.002557 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd74f3-26b4-47eb-b333-2f6db0224c70 None admin] RBAC: Adding query filter params (name=member) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:38.002865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd74f3-26b4-47eb-b333-2f6db0224c70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:38.004272 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd74f3-26b4-47eb-b333-2f6db0224c70 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:38.014340 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 80/150] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:38.015116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2773aee-5bde-4bb7-8166-2f74b0f13488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:38.016360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2773aee-5bde-4bb7-8166-2f74b0f13488 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:38.016360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2773aee-5bde-4bb7-8166-2f74b0f13488 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:38.016360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2773aee-5bde-4bb7-8166-2f74b0f13488 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:38.018127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2773aee-5bde-4bb7-8166-2f74b0f13488 None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:38.021999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2773aee-5bde-4bb7-8166-2f74b0f13488 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:38.021999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 71/151] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:37 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:38.023301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd74f3-26b4-47eb-b333-2f6db0224c70 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:38.027138 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2773aee-5bde-4bb7-8166-2f74b0f13488 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:38.048893 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:38.155452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd74f3-26b4-47eb-b333-2f6db0224c70 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:38.193830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc2854-f379-410e-abf1-f87ff2a2de2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:38.194911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc2854-f379-410e-abf1-f87ff2a2de2a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:38.196528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc2854-f379-410e-abf1-f87ff2a2de2a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:38.197208 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc2854-f379-410e-abf1-f87ff2a2de2a None admin] PATH_INFO: `/v3/users/75e20284884f42e58636018d16cc77a4` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:38.199494 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc2854-f379-410e-abf1-f87ff2a2de2a None admin] RBAC: Authorizing `identity:get_user(user_id=75e20284884f42e58636018d16cc77a4)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:38.203884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc2854-f379-410e-abf1-f87ff2a2de2a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:38.206558 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 81/152] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:38 2022] GET /identity/v3/users/75e20284884f42e58636018d16cc77a4 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:38.214595 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:38.216146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc2854-f379-410e-abf1-f87ff2a2de2a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:38.225503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2773aee-5bde-4bb7-8166-2f74b0f13488 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:38.288859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fc201-b3bb-453d-9fab-90f0c030a387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:38.289544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fc201-b3bb-453d-9fab-90f0c030a387 None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:38.289826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fc201-b3bb-453d-9fab-90f0c030a387 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:38.290101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fc201-b3bb-453d-9fab-90f0c030a387 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119/users/47ca8e21943b4d2bb3a3fea8353a5bbe/roles/ebfd821c64dd448cae68d5408ae32993` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:38.293401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fc201-b3bb-453d-9fab-90f0c030a387 None admin] RBAC: Authorizing `identity:create_grant(project_id=22d0a79f59184a36a955aa3a842c3119, user_id=47ca8e21943b4d2bb3a3fea8353a5bbe, role_id=ebfd821c64dd448cae68d5408ae32993)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:38.294898 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fc201-b3bb-453d-9fab-90f0c030a387 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:38.319328 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 72/153] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:39:38 2022] PUT /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119/users/47ca8e21943b4d2bb3a3fea8353a5bbe/roles/ebfd821c64dd448cae68d5408ae32993 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:39:38.350972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:38.375059 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc2854-f379-410e-abf1-f87ff2a2de2a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:38.416603 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983383a-45a7-4733-8497-accccd342f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:38.417249 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983383a-45a7-4733-8497-accccd342f02 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:38.417529 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983383a-45a7-4733-8497-accccd342f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:38.417803 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983383a-45a7-4733-8497-accccd342f02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:38.508633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983383a-45a7-4733-8497-accccd342f02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:38.510172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983383a-45a7-4733-8497-accccd342f02 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:38.614862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:38.635191 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 82/154] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:39:38.640952 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fc201-b3bb-453d-9fab-90f0c030a387 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:38.764880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fc201-b3bb-453d-9fab-90f0c030a387 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:38.803281 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d295a-5073-44c1-9802-2a58e9a6f3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:38.803927 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d295a-5073-44c1-9802-2a58e9a6f3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:38.804207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d295a-5073-44c1-9802-2a58e9a6f3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:38.804534 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d295a-5073-44c1-9802-2a58e9a6f3d2 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:38.805776 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d295a-5073-44c1-9802-2a58e9a6f3d2 None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:38.807300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d295a-5073-44c1-9802-2a58e9a6f3d2 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:38.809264 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 73/155] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:38 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:38.815769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983383a-45a7-4733-8497-accccd342f02 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:38.932492 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983383a-45a7-4733-8497-accccd342f02 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:38.977972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5bef9d-3319-48c1-a709-fbed814b66fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:38.978695 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bef9d-3319-48c1-a709-fbed814b66fe None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:38.978977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bef9d-3319-48c1-a709-fbed814b66fe None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:38.979251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5bef9d-3319-48c1-a709-fbed814b66fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:38.980115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bef9d-3319-48c1-a709-fbed814b66fe None admin] RBAC: Adding query filter params (role.id=0f74302099d9402ba2a922f36cff56ef, scope.project.id=22d0a79f59184a36a955aa3a842c3119, user.id=75e20284884f42e58636018d16cc77a4) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:38.980427 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bef9d-3319-48c1-a709-fbed814b66fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:38.981878 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5bef9d-3319-48c1-a709-fbed814b66fe None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:38.989125 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 83/156] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:38 2022] GET /identity/v3/role_assignments?user.id=75e20284884f42e58636018d16cc77a4&scope.project.id=22d0a79f59184a36a955aa3a842c3119&role.id=0f74302099d9402ba2a922f36cff56ef => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:39.578874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcc70f-16aa-4334-8f07-4a9ea2af4bdd None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:39.579941 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcc70f-16aa-4334-8f07-4a9ea2af4bdd None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:39.579941 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcc70f-16aa-4334-8f07-4a9ea2af4bdd None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:39.581575 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 74/157] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:39.586427 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3de112-bc7b-4fc3-b9b9-2b289efbde9b None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:39.586746 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3de112-bc7b-4fc3-b9b9-2b289efbde9b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:39.587021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3de112-bc7b-4fc3-b9b9-2b289efbde9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:39.609550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-9f3de112-bc7b-4fc3-b9b9-2b289efbde9b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:39.610824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-9f3de112-bc7b-4fc3-b9b9-2b289efbde9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:39.662013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:39.691033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 84/158] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:39.727535 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fcc70f-16aa-4334-8f07-4a9ea2af4bdd None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:39.754277 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:39.846275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fcc70f-16aa-4334-8f07-4a9ea2af4bdd None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:39.903018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd2b27-d57b-4260-ad47-ce7a8b938d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:39.903678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd2b27-d57b-4260-ad47-ce7a8b938d0b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:39.903958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd2b27-d57b-4260-ad47-ce7a8b938d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:39.904233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd2b27-d57b-4260-ad47-ce7a8b938d0b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:39.919444 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd2b27-d57b-4260-ad47-ce7a8b938d0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:39.925036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-a1cd2b27-d57b-4260-ad47-ce7a8b938d0b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:39:39.925036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:39.925036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:39.925036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:39.925036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:39.925036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:39.925036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:39.925036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:39.925036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:39.925036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:39.925036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:39.925036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:39.925036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:39.925036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:39.925036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:39.925036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:39.926033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:39.926033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:39.926033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:39.926033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:39.926033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:39.926033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:39.926033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:39.926033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:39.926033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:39.926033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:39.926033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:39.926033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:39.926033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:39.926033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:39.926033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:39.926033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:39.927240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:39.927240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:39.927240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:39.927240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:39.927240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:39.927240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:39.927240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:39.927240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:39.927240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:39.927240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:39.927240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:39:39.927240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:39.927240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 75/159] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:39:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:39.932199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3de112-bc7b-4fc3-b9b9-2b289efbde9b None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:40.060264 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3de112-bc7b-4fc3-b9b9-2b289efbde9b None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:40.107687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff475942-e958-4ffa-b3fa-a81446071556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:40.108254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff475942-e958-4ffa-b3fa-a81446071556 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:40.108534 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff475942-e958-4ffa-b3fa-a81446071556 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:40.108802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff475942-e958-4ffa-b3fa-a81446071556 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:40.109613 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff475942-e958-4ffa-b3fa-a81446071556 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:40.109913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff475942-e958-4ffa-b3fa-a81446071556 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:40.111937 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff475942-e958-4ffa-b3fa-a81446071556 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:40.124826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 85/160] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:39:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:40.131097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd2b27-d57b-4260-ad47-ce7a8b938d0b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:40.237971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:40.265120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd2b27-d57b-4260-ad47-ce7a8b938d0b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:40.311193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85dea2-5b3e-444b-a5d4-58d84e1f1f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:40.311858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85dea2-5b3e-444b-a5d4-58d84e1f1f66 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:40.312146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85dea2-5b3e-444b-a5d4-58d84e1f1f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:40.312422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85dea2-5b3e-444b-a5d4-58d84e1f1f66 None admin] PATH_INFO: `/v3/users/75e20284884f42e58636018d16cc77a4` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:40.314248 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85dea2-5b3e-444b-a5d4-58d84e1f1f66 None admin] RBAC: Authorizing `identity:get_user(user_id=75e20284884f42e58636018d16cc77a4)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:40.315740 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85dea2-5b3e-444b-a5d4-58d84e1f1f66 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:40.317663 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 76/161] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:40 2022] GET /identity/v3/users/75e20284884f42e58636018d16cc77a4 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:40.323228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff475942-e958-4ffa-b3fa-a81446071556 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:40.386328 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:40.469497 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff475942-e958-4ffa-b3fa-a81446071556 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:40.512120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45c22b-ab28-4b73-b3d7-f8cc6b4e9d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:40.512338 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2bba1-123d-44a3-82d0-39f5f3cde43e None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:40.512582 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2bba1-123d-44a3-82d0-39f5f3cde43e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:40.512742 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45c22b-ab28-4b73-b3d7-f8cc6b4e9d16 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:40.512852 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d2bba1-123d-44a3-82d0-39f5f3cde43e None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:40.513007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45c22b-ab28-4b73-b3d7-f8cc6b4e9d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:40.513331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45c22b-ab28-4b73-b3d7-f8cc6b4e9d16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:40.514318 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 77/162] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:39:40.521856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a793005-701d-4410-9065-b22ed091863f None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:40.522145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a793005-701d-4410-9065-b22ed091863f None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:40.522427 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a793005-701d-4410-9065-b22ed091863f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:40.563576 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-3a793005-701d-4410-9065-b22ed091863f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:40.564807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-3a793005-701d-4410-9065-b22ed091863f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:40.638466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45c22b-ab28-4b73-b3d7-f8cc6b4e9d16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:40.643834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45c22b-ab28-4b73-b3d7-f8cc6b4e9d16 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:40.680844 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 78/163] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:40.712706 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:40.740871 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a793005-701d-4410-9065-b22ed091863f None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:40.807366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 86/164] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:39:40.813737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45c22b-ab28-4b73-b3d7-f8cc6b4e9d16 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:40.869327 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:40.909352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a793005-701d-4410-9065-b22ed091863f None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:40.966469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b5a2b-20eb-49ed-947e-fd2a57fa6436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:40.967151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b5a2b-20eb-49ed-947e-fd2a57fa6436 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:40.967687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b5a2b-20eb-49ed-947e-fd2a57fa6436 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:40.967966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8b5a2b-20eb-49ed-947e-fd2a57fa6436 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:40.991813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8b5a2b-20eb-49ed-947e-fd2a57fa6436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:40.997826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-1d8b5a2b-20eb-49ed-947e-fd2a57fa6436 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:40.997826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:40.997826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:40.997826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:40.997826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:40.997826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:40.997826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:40.997826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:40.997826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:40.997826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:40.997826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:40.997826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:40.997826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:40.997826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:40.997826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:40.997826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:40.998733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:40.998733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:40.998733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:40.998733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:40.998733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:40.998733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:40.998733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:40.998733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:40.998733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:40.998733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:40.998733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:40.998733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:40.998733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:40.998733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:40.998733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:40.998733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:40.999507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:40.999507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:40.999507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:40.999507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:40.999507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:40.999507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:40.999507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:40.999507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:40.999507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:40.999507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:40.999507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:40.999507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:41.000375 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 79/165] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:41.005119 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b5a2b-20eb-49ed-947e-fd2a57fa6436 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:41.013529 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45c22b-ab28-4b73-b3d7-f8cc6b4e9d16 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:41.087206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0d09f-bd2b-4054-af67-3bf306330f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:41.087860 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0d09f-bd2b-4054-af67-3bf306330f6c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:41.088137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0d09f-bd2b-4054-af67-3bf306330f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:41.088408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0d09f-bd2b-4054-af67-3bf306330f6c None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:41.091485 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0d09f-bd2b-4054-af67-3bf306330f6c None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:41.093823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0d09f-bd2b-4054-af67-3bf306330f6c None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:41.098568 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 87/166] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:40 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:41.099890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6414f-c3ed-4156-9220-b67174650298 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:41.100952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6414f-c3ed-4156-9220-b67174650298 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:41.101532 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6414f-c3ed-4156-9220-b67174650298 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:41.104121 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 88/167] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:41.105009 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6414f-c3ed-4156-9220-b67174650298 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:41.122877 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:41.185615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8b5a2b-20eb-49ed-947e-fd2a57fa6436 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:41.260099 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185c3f2-2e67-46f5-ab8a-a4d2a05cdf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:41.260968 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185c3f2-2e67-46f5-ab8a-a4d2a05cdf64 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:41.261423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185c3f2-2e67-46f5-ab8a-a4d2a05cdf64 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:41.261838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185c3f2-2e67-46f5-ab8a-a4d2a05cdf64 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:41.262745 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185c3f2-2e67-46f5-ab8a-a4d2a05cdf64 None admin] RBAC: Adding query filter params (name=member) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:41.263188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185c3f2-2e67-46f5-ab8a-a4d2a05cdf64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:41.264801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185c3f2-2e67-46f5-ab8a-a4d2a05cdf64 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:41.281435 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 80/168] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:41.284997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffa8a1-970a-497c-abf9-57e9faac51ea None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:41.288993 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffa8a1-970a-497c-abf9-57e9faac51ea None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:41.289430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ffa8a1-970a-497c-abf9-57e9faac51ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:41.301203 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:41.318917 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6414f-c3ed-4156-9220-b67174650298 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:41.325054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-72ffa8a1-970a-497c-abf9-57e9faac51ea None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:41.326261 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-72ffa8a1-970a-497c-abf9-57e9faac51ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:41.367219 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99a97c-4528-4e9a-8aec-7010f8601c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:41.368079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99a97c-4528-4e9a-8aec-7010f8601c92 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:41.368603 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99a97c-4528-4e9a-8aec-7010f8601c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:41.369016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99a97c-4528-4e9a-8aec-7010f8601c92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:41.369964 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99a97c-4528-4e9a-8aec-7010f8601c92 None admin] RBAC: Adding query filter params (role.id=79ff845a41e241229433560e2b0d6fe5, scope.project.id=22d0a79f59184a36a955aa3a842c3119, user.id=75e20284884f42e58636018d16cc77a4) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:41.370414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99a97c-4528-4e9a-8aec-7010f8601c92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:41.372022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99a97c-4528-4e9a-8aec-7010f8601c92 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:41.378847 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 89/169] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:41 2022] GET /identity/v3/role_assignments?user.id=75e20284884f42e58636018d16cc77a4&scope.project.id=22d0a79f59184a36a955aa3a842c3119&role.id=79ff845a41e241229433560e2b0d6fe5 => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:41.379877 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99a97c-4528-4e9a-8aec-7010f8601c92 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:41.451489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 81/170] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:41.453831 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:41.487585 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99a97c-4528-4e9a-8aec-7010f8601c92 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:41.489403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ffa8a1-970a-497c-abf9-57e9faac51ea None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:41.524785 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665384e-99b8-42d9-9780-4086483f17d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:41.525712 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665384e-99b8-42d9-9780-4086483f17d2 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:41.526158 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665384e-99b8-42d9-9780-4086483f17d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:41.526602 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665384e-99b8-42d9-9780-4086483f17d2 None admin] PATH_INFO: `/v3/users/75e20284884f42e58636018d16cc77a4` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:41.528050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665384e-99b8-42d9-9780-4086483f17d2 None admin] RBAC: Authorizing `identity:get_user(user_id=75e20284884f42e58636018d16cc77a4)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:41.529700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665384e-99b8-42d9-9780-4086483f17d2 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:41.531606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 90/171] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:41 2022] GET /identity/v3/users/75e20284884f42e58636018d16cc77a4 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:41.533038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18df6d-3e66-43c6-b46c-d6a5dbbecb67 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:41.533480 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18df6d-3e66-43c6-b46c-d6a5dbbecb67 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:41.533896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18df6d-3e66-43c6-b46c-d6a5dbbecb67 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:41.535305 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 91/172] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:41.543167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a735a-ec88-4bfe-93b4-9154e7c2d0a0 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:41.543661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a735a-ec88-4bfe-93b4-9154e7c2d0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:41.544078 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a735a-ec88-4bfe-93b4-9154e7c2d0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:41.565930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-ee9a735a-ec88-4bfe-93b4-9154e7c2d0a0 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:41.567685 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ee9a735a-ec88-4bfe-93b4-9154e7c2d0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:41.647206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 92/173] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:41.648407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a735a-ec88-4bfe-93b4-9154e7c2d0a0 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:41.720868 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ffa8a1-970a-497c-abf9-57e9faac51ea None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:41.724761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:41.761470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a735a-ec88-4bfe-93b4-9154e7c2d0a0 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:41.772406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:41.778614 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ad2c5-0887-488a-b4e4-49f904af6540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:41.779375 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ad2c5-0887-488a-b4e4-49f904af6540 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:41.779758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ad2c5-0887-488a-b4e4-49f904af6540 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:41.779921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ad2c5-0887-488a-b4e4-49f904af6540 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:41.795712 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ad2c5-0887-488a-b4e4-49f904af6540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:41.804847 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fa149-3741-4921-9dbe-c63ac21f8996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:41.805485 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fa149-3741-4921-9dbe-c63ac21f8996 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:41.805764 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fa149-3741-4921-9dbe-c63ac21f8996 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:41.806026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fa149-3741-4921-9dbe-c63ac21f8996 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:41.810693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-6e1ad2c5-0887-488a-b4e4-49f904af6540 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:41.810693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:41.810693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:41.810693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:41.810693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:41.810693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:41.810693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:41.810693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:41.810693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:41.810693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:41.810693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:41.810693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:41.810693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:41.810693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:41.810693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:41.810693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:41.811591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:41.811591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:41.811591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:41.811591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:41.811591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:41.811591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:41.811591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:41.811591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:41.811591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:41.811591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:41.811591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:41.811591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:41.811591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:41.811591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:41.811591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:41.811591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:41.812393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:41.812393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:41.812393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:41.812393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:41.812393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:41.812393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:41.812393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:41.812393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:41.812393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:41.812393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:41.812393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:41.812393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:41.812393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 82/174] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:39:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:39:41.815334 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ad2c5-0887-488a-b4e4-49f904af6540 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:41.908105 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8fa149-3741-4921-9dbe-c63ac21f8996 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:41.909568 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8fa149-3741-4921-9dbe-c63ac21f8996 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:41.976030 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ad2c5-0887-488a-b4e4-49f904af6540 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:41.994924 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:42.025115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e823610-b92d-4c04-9ed4-4f6379646e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:42.026054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e823610-b92d-4c04-9ed4-4f6379646e9b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:42.026784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e823610-b92d-4c04-9ed4-4f6379646e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:42.027637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e823610-b92d-4c04-9ed4-4f6379646e9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:42.042547 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 93/175] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:39:42.044804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fa149-3741-4921-9dbe-c63ac21f8996 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:42.049167 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e823610-b92d-4c04-9ed4-4f6379646e9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:42.056157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-2e823610-b92d-4c04-9ed4-4f6379646e9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:42.056157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:42.056157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:42.056157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:42.056157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:42.056157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:42.056157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:42.056157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:42.056157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:42.056157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:42.056157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:42.056157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:42.056157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:42.056157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:42.056157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:42.056157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:42.057051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:42.057051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:42.057051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:42.057051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:42.057051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:42.057051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:42.057051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:42.057051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:42.057051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:42.057051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:42.057051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:42.057051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:42.057051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:42.057051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:42.057051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:42.057051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:42.057888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:42.057888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:42.057888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:42.057888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:42.057888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:42.057888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:42.057888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:42.057888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:42.057888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:42.057888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:42.057888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:42.057888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:42.061907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 83/176] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:39:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:42.062954 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a57448-6788-470f-8c3a-e9615fec90a3 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:42.063500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a57448-6788-470f-8c3a-e9615fec90a3 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:42.063955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a57448-6788-470f-8c3a-e9615fec90a3 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:42.065580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 84/177] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:42.066532 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a57448-6788-470f-8c3a-e9615fec90a3 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:42.132139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:42.161423 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fa149-3741-4921-9dbe-c63ac21f8996 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:42.182640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a57448-6788-470f-8c3a-e9615fec90a3 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:42.194963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2688e4b-9b07-4494-a5a5-5a19c2617207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:42.195571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2688e4b-9b07-4494-a5a5-5a19c2617207 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:42.195851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2688e4b-9b07-4494-a5a5-5a19c2617207 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:42.196120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2688e4b-9b07-4494-a5a5-5a19c2617207 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:42.196836 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2688e4b-9b07-4494-a5a5-5a19c2617207 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:42.197145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2688e4b-9b07-4494-a5a5-5a19c2617207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:42.198536 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2688e4b-9b07-4494-a5a5-5a19c2617207 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:42.206607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 94/178] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:42.207465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2688e4b-9b07-4494-a5a5-5a19c2617207 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:42.216783 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7cc29-c55e-43a3-a4c4-e840e38b7c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:42.217422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7cc29-c55e-43a3-a4c4-e840e38b7c64 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:42.217696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7cc29-c55e-43a3-a4c4-e840e38b7c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:42.217965 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7cc29-c55e-43a3-a4c4-e840e38b7c64 None admin] PATH_INFO: `/v3/projects/28eb7ee742aa442c8b9db488ae0869b2` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:42.219134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7cc29-c55e-43a3-a4c4-e840e38b7c64 None admin] RBAC: Authorizing `identity:get_project(project_id=28eb7ee742aa442c8b9db488ae0869b2)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:42.221079 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7cc29-c55e-43a3-a4c4-e840e38b7c64 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:42.222741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 85/179] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:42 2022] GET /identity/v3/projects/28eb7ee742aa442c8b9db488ae0869b2 => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:42.227675 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47fdb7-3e63-40e3-8fe1-311fb60445a0 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:42.227900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47fdb7-3e63-40e3-8fe1-311fb60445a0 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:42.228178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47fdb7-3e63-40e3-8fe1-311fb60445a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:42.238656 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:42.245886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-6f47fdb7-3e63-40e3-8fe1-311fb60445a0 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:42.247081 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6f47fdb7-3e63-40e3-8fe1-311fb60445a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:42.306602 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2688e4b-9b07-4494-a5a5-5a19c2617207 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:42.310577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 86/180] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:42.311774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ce948-8e81-438d-acfd-848afa59c70f None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:42.312055 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ce948-8e81-438d-acfd-848afa59c70f None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:42.312325 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ce948-8e81-438d-acfd-848afa59c70f None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:42.313624 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 87/181] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:42.317851 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbd3f1-1b8f-4335-83f2-bb23a9f00fef None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:42.317851 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbd3f1-1b8f-4335-83f2-bb23a9f00fef None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:42.317851 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbd3f1-1b8f-4335-83f2-bb23a9f00fef None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:42.317851 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 88/182] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:42.317851 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbd3f1-1b8f-4335-83f2-bb23a9f00fef None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:42.342614 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cbbc4-256b-477c-bcde-0ae037b40d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:42.343241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cbbc4-256b-477c-bcde-0ae037b40d17 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:42.343523 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cbbc4-256b-477c-bcde-0ae037b40d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:42.343800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cbbc4-256b-477c-bcde-0ae037b40d17 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:42.344522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366cbbc4-256b-477c-bcde-0ae037b40d17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:42.345373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366cbbc4-256b-477c-bcde-0ae037b40d17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:42.346816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366cbbc4-256b-477c-bcde-0ae037b40d17 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:42.358995 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 95/183] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:42.358995 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cbbc4-256b-477c-bcde-0ae037b40d17 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:42.378814 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:42.428717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbd3f1-1b8f-4335-83f2-bb23a9f00fef None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:42.443251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:42.453238 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cbbc4-256b-477c-bcde-0ae037b40d17 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:42.461857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af435a4-1bca-4ed8-952a-9124952107f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:42.462500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af435a4-1bca-4ed8-952a-9124952107f8 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:42.462781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af435a4-1bca-4ed8-952a-9124952107f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:42.463049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af435a4-1bca-4ed8-952a-9124952107f8 None admin] PATH_INFO: `/v3/users/f665e42501ed48eda8620c2548069351` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:42.464205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af435a4-1bca-4ed8-952a-9124952107f8 None admin] RBAC: Authorizing `identity:get_user(user_id=f665e42501ed48eda8620c2548069351)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:42.465629 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af435a4-1bca-4ed8-952a-9124952107f8 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:42.467235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 89/184] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:42 2022] GET /identity/v3/users/f665e42501ed48eda8620c2548069351 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:42.468327 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f86241-6d80-42b2-99a1-3513610139ff None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:42.468591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f86241-6d80-42b2-99a1-3513610139ff None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:42.468861 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f86241-6d80-42b2-99a1-3513610139ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:42.486665 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-d4f86241-6d80-42b2-99a1-3513610139ff None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:42.486801 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb562d-518e-400e-83e8-6da8f9401f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:42.487399 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb562d-518e-400e-83e8-6da8f9401f51 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:42.487680 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb562d-518e-400e-83e8-6da8f9401f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:42.487790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d4f86241-6d80-42b2-99a1-3513610139ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:42.487952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb562d-518e-400e-83e8-6da8f9401f51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:42.488740 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb562d-518e-400e-83e8-6da8f9401f51 None admin] RBAC: Adding query filter params (role.id=0f74302099d9402ba2a922f36cff56ef, scope.project.id=28eb7ee742aa442c8b9db488ae0869b2, user.id=75e20284884f42e58636018d16cc77a4) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:42.489045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb562d-518e-400e-83e8-6da8f9401f51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:42.491339 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb562d-518e-400e-83e8-6da8f9401f51 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:42.497773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 96/185] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:42 2022] GET /identity/v3/role_assignments?user.id=75e20284884f42e58636018d16cc77a4&scope.project.id=28eb7ee742aa442c8b9db488ae0869b2&role.id=0f74302099d9402ba2a922f36cff56ef => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:42.499307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b11862-cffe-486f-b090-b2935418a396 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:42.499594 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b11862-cffe-486f-b090-b2935418a396 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:42.499867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b11862-cffe-486f-b090-b2935418a396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:42.528038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-e6b11862-cffe-486f-b090-b2935418a396 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:42.529293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e6b11862-cffe-486f-b090-b2935418a396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:42.574510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 90/186] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:42.574510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f86241-6d80-42b2-99a1-3513610139ff None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:42.615276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 97/187] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:42.616227 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b11862-cffe-486f-b090-b2935418a396 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:42.637941 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:42.690636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f86241-6d80-42b2-99a1-3513610139ff None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:42.716245 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:42.733842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b11862-cffe-486f-b090-b2935418a396 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:42.738688 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4886d1a-df8a-4b15-b439-06213cd7884a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:42.739281 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4886d1a-df8a-4b15-b439-06213cd7884a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:42.739557 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4886d1a-df8a-4b15-b439-06213cd7884a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:42.739826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4886d1a-df8a-4b15-b439-06213cd7884a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:42.749923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4886d1a-df8a-4b15-b439-06213cd7884a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:42.755404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-d4886d1a-df8a-4b15-b439-06213cd7884a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:39:42.755404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:42.755404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:42.755404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:42.755404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:42.755404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:42.755404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:42.755404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:42.755404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:42.755404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:42.755404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:42.755404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:42.755404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:42.755404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:42.755404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:42.755404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:42.756475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:42.756475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:42.756475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:42.756475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:42.756475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:42.756475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:42.756475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:42.756475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:42.756475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:42.756475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:42.756475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:42.756475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:42.756475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:42.756475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:42.756475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:42.756475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:42.757367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:42.757367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:42.757367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:42.757367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:42.757367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:42.757367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:42.757367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:42.757367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:42.757367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:42.757367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:42.757367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:39:42.757367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:42.757367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 91/188] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:39:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:42.757367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4886d1a-df8a-4b15-b439-06213cd7884a None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:42.771111 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab0c554-df81-4f25-99d8-58db4ef4dba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:42.771750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0c554-df81-4f25-99d8-58db4ef4dba3 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:42.772031 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0c554-df81-4f25-99d8-58db4ef4dba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:42.772301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab0c554-df81-4f25-99d8-58db4ef4dba3 None admin] PATH_INFO: `/v3/users/c9e3cde4b4a04a5c9d2f82a577a31030` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:42.785502 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab0c554-df81-4f25-99d8-58db4ef4dba3 None admin] RBAC: Authorizing `identity:get_user(user_id=c9e3cde4b4a04a5c9d2f82a577a31030)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:42.786987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab0c554-df81-4f25-99d8-58db4ef4dba3 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:42.788600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 98/189] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:42 2022] GET /identity/v3/users/c9e3cde4b4a04a5c9d2f82a577a31030 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:42.793762 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab0c554-df81-4f25-99d8-58db4ef4dba3 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:42.859566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4886d1a-df8a-4b15-b439-06213cd7884a None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:42.869121 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:42.895015 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab0c554-df81-4f25-99d8-58db4ef4dba3 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:42.895864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf557e3-090d-4a78-b1f2-fd7a268a2b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:42.896483 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf557e3-090d-4a78-b1f2-fd7a268a2b8e None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:42.896762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf557e3-090d-4a78-b1f2-fd7a268a2b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:42.897036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf557e3-090d-4a78-b1f2-fd7a268a2b8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:42.928031 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837dbe0-7052-48df-9441-070408a2ea08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:42.928656 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837dbe0-7052-48df-9441-070408a2ea08 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:42.928930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837dbe0-7052-48df-9441-070408a2ea08 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:42.929198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837dbe0-7052-48df-9441-070408a2ea08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:42.930333 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837dbe0-7052-48df-9441-070408a2ea08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:42.931786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837dbe0-7052-48df-9441-070408a2ea08 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:42.933518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 99/190] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:42.934095 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837dbe0-7052-48df-9441-070408a2ea08 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:42.948592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:42.965524 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf557e3-090d-4a78-b1f2-fd7a268a2b8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:42.967002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf557e3-090d-4a78-b1f2-fd7a268a2b8e None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:43.031394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837dbe0-7052-48df-9441-070408a2ea08 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:43.064366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 92/191] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:39:43.064525 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d3d3e-a47a-44cd-909e-9ffc1e9062e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:43.065058 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d3d3e-a47a-44cd-909e-9ffc1e9062e2 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:43.065219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf557e3-090d-4a78-b1f2-fd7a268a2b8e None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:43.065343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d3d3e-a47a-44cd-909e-9ffc1e9062e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:43.065592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d3d3e-a47a-44cd-909e-9ffc1e9062e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:43.075298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:43.075781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d3d3e-a47a-44cd-909e-9ffc1e9062e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:43.081388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-5d0d3d3e-a47a-44cd-909e-9ffc1e9062e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:39:43.081388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:43.081388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:43.081388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:43.081388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:43.081388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:43.081388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:43.081388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:43.081388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:43.081388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:43.081388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:43.081388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:43.081388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:43.081388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:43.081388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:43.081388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:43.082295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:43.082295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:43.082295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:43.082295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:43.082295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:43.082295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:43.082295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:43.082295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:43.082295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:43.082295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:43.082295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:43.082295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:43.082295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:43.082295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:43.082295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:43.082295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:43.083136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:43.083136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:43.083136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:43.083136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:43.083136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:43.083136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:43.083136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:43.083136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:43.083136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:43.083136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:43.083136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:39:43.083136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:39:43.083136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 100/192] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:43.083828 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d3d3e-a47a-44cd-909e-9ffc1e9062e2 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:43.150075 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:43.166028 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf557e3-090d-4a78-b1f2-fd7a268a2b8e None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:43.181007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d3d3e-a47a-44cd-909e-9ffc1e9062e2 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:43.200300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b72d705-2633-46ef-bd96-2d592b72974b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:43.200910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72d705-2633-46ef-bd96-2d592b72974b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:43.201183 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72d705-2633-46ef-bd96-2d592b72974b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:43.201451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72d705-2633-46ef-bd96-2d592b72974b None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:43.202167 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b72d705-2633-46ef-bd96-2d592b72974b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:43.202466 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b72d705-2633-46ef-bd96-2d592b72974b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:43.204831 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b72d705-2633-46ef-bd96-2d592b72974b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:43.211808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 93/193] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:39:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:43.212563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b72d705-2633-46ef-bd96-2d592b72974b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:43.219028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58a41f-2be8-4b36-91fb-c25854433fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:43.219653 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58a41f-2be8-4b36-91fb-c25854433fae None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:43.219962 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58a41f-2be8-4b36-91fb-c25854433fae None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:43.220235 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e58a41f-2be8-4b36-91fb-c25854433fae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:43.283552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:43.285967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58a41f-2be8-4b36-91fb-c25854433fae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:43.287458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e58a41f-2be8-4b36-91fb-c25854433fae None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:43.315634 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b72d705-2633-46ef-bd96-2d592b72974b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:43.347818 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e92935-1aea-46de-91ce-d0931d66283e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:43.348424 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e92935-1aea-46de-91ce-d0931d66283e None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:43.348704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e92935-1aea-46de-91ce-d0931d66283e None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:43.348976 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e92935-1aea-46de-91ce-d0931d66283e None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:43.349659 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e92935-1aea-46de-91ce-d0931d66283e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:43.351128 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e92935-1aea-46de-91ce-d0931d66283e None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:43.354098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:43.379381 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 94/194] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:39:43 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:39:43.380190 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e92935-1aea-46de-91ce-d0931d66283e None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:43.394619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 101/195] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:39:43.395737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58a41f-2be8-4b36-91fb-c25854433fae None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:43.503840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e92935-1aea-46de-91ce-d0931d66283e None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:43.508986 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e58a41f-2be8-4b36-91fb-c25854433fae None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:43.535269 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:43.545793 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d7244-8bb1-4e42-a5d5-92cfb4eb23dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:43.546715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d7244-8bb1-4e42-a5d5-92cfb4eb23dc None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:43.547163 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d7244-8bb1-4e42-a5d5-92cfb4eb23dc None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:43.547577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d7244-8bb1-4e42-a5d5-92cfb4eb23dc None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:43.549476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d7244-8bb1-4e42-a5d5-92cfb4eb23dc None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:43.551125 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d7244-8bb1-4e42-a5d5-92cfb4eb23dc None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:43.552962 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 95/196] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:43 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:43.553863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d7244-8bb1-4e42-a5d5-92cfb4eb23dc None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:43.555823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038302e-97c4-45d7-abde-f59408fbc3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:43.556448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038302e-97c4-45d7-abde-f59408fbc3db None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:43.556732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038302e-97c4-45d7-abde-f59408fbc3db None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:43.557002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038302e-97c4-45d7-abde-f59408fbc3db None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:43.557718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038302e-97c4-45d7-abde-f59408fbc3db None admin] RBAC: Adding query filter params (name=reader) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:43.558019 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038302e-97c4-45d7-abde-f59408fbc3db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:43.559522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038302e-97c4-45d7-abde-f59408fbc3db None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:43.569115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 102/197] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:43.569973 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038302e-97c4-45d7-abde-f59408fbc3db None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:43.611214 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:43.680356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d7244-8bb1-4e42-a5d5-92cfb4eb23dc None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:43.693116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038302e-97c4-45d7-abde-f59408fbc3db None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:43.719564 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad0641-e48c-4375-b9eb-ff7df10e4787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:43.720741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad0641-e48c-4375-b9eb-ff7df10e4787 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:43.721022 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad0641-e48c-4375-b9eb-ff7df10e4787 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:43.721347 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad0641-e48c-4375-b9eb-ff7df10e4787 None admin] PATH_INFO: `/v3/users/c9e3cde4b4a04a5c9d2f82a577a31030` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:43.722577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad0641-e48c-4375-b9eb-ff7df10e4787 None admin] RBAC: Authorizing `identity:get_user(user_id=c9e3cde4b4a04a5c9d2f82a577a31030)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:43.724169 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad0641-e48c-4375-b9eb-ff7df10e4787 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:43.725799 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 96/198] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:43 2022] GET /identity/v3/users/c9e3cde4b4a04a5c9d2f82a577a31030 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:43.726601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad0641-e48c-4375-b9eb-ff7df10e4787 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:43.756859 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ea65d-f40c-4404-809b-ef3645599a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:43.757559 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ea65d-f40c-4404-809b-ef3645599a76 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:43.757899 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ea65d-f40c-4404-809b-ef3645599a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:43.758173 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ea65d-f40c-4404-809b-ef3645599a76 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:43.759390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ea65d-f40c-4404-809b-ef3645599a76 None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:43.760823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ea65d-f40c-4404-809b-ef3645599a76 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:43.762870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 103/199] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:43 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:43.763703 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ea65d-f40c-4404-809b-ef3645599a76 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:43.802968 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:43.842887 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad0641-e48c-4375-b9eb-ff7df10e4787 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:43.880284 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b02387-18a0-42a2-b8b9-321abbc527ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:43.881186 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b02387-18a0-42a2-b8b9-321abbc527ba None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:43.881699 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b02387-18a0-42a2-b8b9-321abbc527ba None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:43.882177 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b02387-18a0-42a2-b8b9-321abbc527ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:43.883217 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b02387-18a0-42a2-b8b9-321abbc527ba None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=22d0a79f59184a36a955aa3a842c3119, user.id=f665e42501ed48eda8620c2548069351) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:43.883742 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b02387-18a0-42a2-b8b9-321abbc527ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:43.885385 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b02387-18a0-42a2-b8b9-321abbc527ba None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:43.890839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ea65d-f40c-4404-809b-ef3645599a76 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:43.892931 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 97/200] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:43 2022] GET /identity/v3/role_assignments?user.id=f665e42501ed48eda8620c2548069351&scope.project.id=22d0a79f59184a36a955aa3a842c3119&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:43.893501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:43.895188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b02387-18a0-42a2-b8b9-321abbc527ba None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:43.962063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae14bab-4402-42a5-a035-f78231e5750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:43.962758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae14bab-4402-42a5-a035-f78231e5750b None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:43.963101 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae14bab-4402-42a5-a035-f78231e5750b None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:43.963606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae14bab-4402-42a5-a035-f78231e5750b None admin] PATH_INFO: `/v3/users/47ca8e21943b4d2bb3a3fea8353a5bbe` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:43.964565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae14bab-4402-42a5-a035-f78231e5750b None admin] RBAC: Authorizing `identity:get_user(user_id=47ca8e21943b4d2bb3a3fea8353a5bbe)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:43.966047 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae14bab-4402-42a5-a035-f78231e5750b None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:43.967660 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 104/201] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:43 2022] GET /identity/v3/users/47ca8e21943b4d2bb3a3fea8353a5bbe => generated 289 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:43.968421 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae14bab-4402-42a5-a035-f78231e5750b None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:44.010631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b02387-18a0-42a2-b8b9-321abbc527ba None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:44.050891 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:44.054353 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d289d-b3ab-4205-8526-6186f4bd9b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:44.055005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d289d-b3ab-4205-8526-6186f4bd9b55 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:44.055286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d289d-b3ab-4205-8526-6186f4bd9b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:44.055558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d289d-b3ab-4205-8526-6186f4bd9b55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:44.130367 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae14bab-4402-42a5-a035-f78231e5750b None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:44.188117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5285d14-b2a5-4799-af44-a703a547d8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:44.189277 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5285d14-b2a5-4799-af44-a703a547d8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:44.189563 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5285d14-b2a5-4799-af44-a703a547d8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:44.189841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5285d14-b2a5-4799-af44-a703a547d8f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:44.190687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5285d14-b2a5-4799-af44-a703a547d8f0 None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=efec6052310d4c7289f0ce48ac293400, user.id=c9e3cde4b4a04a5c9d2f82a577a31030) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:44.190872 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d289d-b3ab-4205-8526-6186f4bd9b55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:44.190998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5285d14-b2a5-4799-af44-a703a547d8f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:44.192429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5285d14-b2a5-4799-af44-a703a547d8f0 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:44.192722 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428d289d-b3ab-4205-8526-6186f4bd9b55 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:44.200163 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 105/202] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:43 2022] GET /identity/v3/role_assignments?user.id=c9e3cde4b4a04a5c9d2f82a577a31030&scope.project.id=efec6052310d4c7289f0ce48ac293400&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:44.200876 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5285d14-b2a5-4799-af44-a703a547d8f0 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:44.268504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:44.400837 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 98/203] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:39:44.404636 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5285d14-b2a5-4799-af44-a703a547d8f0 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:44.408113 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d289d-b3ab-4205-8526-6186f4bd9b55 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:44.443393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:44.467242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650ffcc-b463-4d47-8e43-a64f8178748e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:44.467242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650ffcc-b463-4d47-8e43-a64f8178748e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:44.467523 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650ffcc-b463-4d47-8e43-a64f8178748e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:44.467741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650ffcc-b463-4d47-8e43-a64f8178748e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:44.558072 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650ffcc-b463-4d47-8e43-a64f8178748e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:44.560074 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650ffcc-b463-4d47-8e43-a64f8178748e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:44.562828 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d289d-b3ab-4205-8526-6186f4bd9b55 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:44.602805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:44.621119 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049edab-2dcb-4457-8eff-f5ff44e24163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:44.621956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049edab-2dcb-4457-8eff-f5ff44e24163 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:44.622363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049edab-2dcb-4457-8eff-f5ff44e24163 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:44.622708 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049edab-2dcb-4457-8eff-f5ff44e24163 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:44.627495 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7049edab-2dcb-4457-8eff-f5ff44e24163 None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:44.628806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7049edab-2dcb-4457-8eff-f5ff44e24163 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:44.630628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 99/204] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:44 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:44.636328 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049edab-2dcb-4457-8eff-f5ff44e24163 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:44.682100 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 106/205] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:39:44.687781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650ffcc-b463-4d47-8e43-a64f8178748e None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:44.802427 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:44.812174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049edab-2dcb-4457-8eff-f5ff44e24163 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:44.879624 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfccbb9-ec8d-4993-8a92-f561f0d4300b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:44.880361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfccbb9-ec8d-4993-8a92-f561f0d4300b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:44.880579 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfccbb9-ec8d-4993-8a92-f561f0d4300b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:44.880772 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfccbb9-ec8d-4993-8a92-f561f0d4300b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:44.881573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfccbb9-ec8d-4993-8a92-f561f0d4300b None admin] RBAC: Adding query filter params (role.id=79ff845a41e241229433560e2b0d6fe5, scope.project.id=efec6052310d4c7289f0ce48ac293400, user.id=c9e3cde4b4a04a5c9d2f82a577a31030) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:44.881877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfccbb9-ec8d-4993-8a92-f561f0d4300b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:44.883367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfccbb9-ec8d-4993-8a92-f561f0d4300b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:44.886924 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650ffcc-b463-4d47-8e43-a64f8178748e None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:44.897626 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 100/206] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:44 2022] GET /identity/v3/role_assignments?user.id=c9e3cde4b4a04a5c9d2f82a577a31030&scope.project.id=efec6052310d4c7289f0ce48ac293400&role.id=79ff845a41e241229433560e2b0d6fe5 => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:44.926563 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed93a95-3f64-4d5c-ace1-44caff0e8d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:44.927232 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed93a95-3f64-4d5c-ace1-44caff0e8d78 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:44.927511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed93a95-3f64-4d5c-ace1-44caff0e8d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:44.927783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed93a95-3f64-4d5c-ace1-44caff0e8d78 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:44.929031 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed93a95-3f64-4d5c-ace1-44caff0e8d78 None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:44.930499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed93a95-3f64-4d5c-ace1-44caff0e8d78 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:44.932642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 107/207] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:44 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:44.938673 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfccbb9-ec8d-4993-8a92-f561f0d4300b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:45.073320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfccbb9-ec8d-4993-8a92-f561f0d4300b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:45.124129 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a966d4e-5ced-44a5-a6cf-9c1d722797ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:45.124751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a966d4e-5ced-44a5-a6cf-9c1d722797ff None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:45.125032 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a966d4e-5ced-44a5-a6cf-9c1d722797ff None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:45.125307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a966d4e-5ced-44a5-a6cf-9c1d722797ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:45.126142 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a966d4e-5ced-44a5-a6cf-9c1d722797ff None admin] RBAC: Adding query filter params (role.id=ebfd821c64dd448cae68d5408ae32993, scope.project.id=22d0a79f59184a36a955aa3a842c3119, user.id=47ca8e21943b4d2bb3a3fea8353a5bbe) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:45.126456 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a966d4e-5ced-44a5-a6cf-9c1d722797ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:45.127920 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a966d4e-5ced-44a5-a6cf-9c1d722797ff None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:45.133799 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 101/208] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:44 2022] GET /identity/v3/role_assignments?user.id=47ca8e21943b4d2bb3a3fea8353a5bbe&scope.project.id=22d0a79f59184a36a955aa3a842c3119&role.id=ebfd821c64dd448cae68d5408ae32993 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:45.617438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0778775-0c79-4110-9a13-bf79ae0840d5 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:45.617806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0778775-0c79-4110-9a13-bf79ae0840d5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:45.617984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0778775-0c79-4110-9a13-bf79ae0840d5 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:45.619606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 108/209] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:45.624950 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a030905-2282-409d-8ac6-281fcabca2f2 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:45.625244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a030905-2282-409d-8ac6-281fcabca2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:45.625519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a030905-2282-409d-8ac6-281fcabca2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:45.644792 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-3a030905-2282-409d-8ac6-281fcabca2f2 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:45.645996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-3a030905-2282-409d-8ac6-281fcabca2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:45.664104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:45.723068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 102/210] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:39:45.746678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0778775-0c79-4110-9a13-bf79ae0840d5 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:45.749675 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:45.863677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0778775-0c79-4110-9a13-bf79ae0840d5 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:45.904337 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c146f75-0139-4d9d-914d-6625d02e277b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:45.905045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c146f75-0139-4d9d-914d-6625d02e277b None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:45.905262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c146f75-0139-4d9d-914d-6625d02e277b None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:45.905538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c146f75-0139-4d9d-914d-6625d02e277b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:45.916074 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c146f75-0139-4d9d-914d-6625d02e277b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:45.921188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-6c146f75-0139-4d9d-914d-6625d02e277b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:45.921188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:45.921188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:45.921188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:45.921188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:45.921188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:45.921188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:45.921188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:45.921188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:45.921188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:45.921188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:45.921188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:45.921188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:45.921188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:45.921188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:45.921188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:45.922071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:45.922071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:45.922071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:45.922071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:45.922071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:45.922071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:45.922071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:45.922071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:45.922071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:45.922071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:45.922071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:45.922071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:45.922071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:45.922071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:45.922071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:45.922071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:45.922941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:45.922941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:45.922941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:45.922941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:45.922941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:45.922941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:45.922941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:45.922941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:45.922941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:45.922941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:45.922941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:45.922941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:39:45.922941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 109/211] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:45.929594 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a030905-2282-409d-8ac6-281fcabca2f2 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:46.019942 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54007360-b36b-47b7-aa3f-a15b23f08be7 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:46.020253 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54007360-b36b-47b7-aa3f-a15b23f08be7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:46.020624 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54007360-b36b-47b7-aa3f-a15b23f08be7 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:46.022242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 110/212] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:39:46.027223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405bd3b-0705-4e2d-a33c-3353a87fe81b None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:46.027522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405bd3b-0705-4e2d-a33c-3353a87fe81b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:46.027789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405bd3b-0705-4e2d-a33c-3353a87fe81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:46.029738 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a030905-2282-409d-8ac6-281fcabca2f2 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:46.047035 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1405bd3b-0705-4e2d-a33c-3353a87fe81b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:46.048856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1405bd3b-0705-4e2d-a33c-3353a87fe81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:46.067788 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42126e-7ff0-45d4-a3be-100aaa282c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:46.068875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42126e-7ff0-45d4-a3be-100aaa282c2d None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:46.069360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42126e-7ff0-45d4-a3be-100aaa282c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:46.070099 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc42126e-7ff0-45d4-a3be-100aaa282c2d None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:46.071087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42126e-7ff0-45d4-a3be-100aaa282c2d None admin] RBAC: Adding query filter params (name=member) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:46.071566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42126e-7ff0-45d4-a3be-100aaa282c2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:46.073548 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc42126e-7ff0-45d4-a3be-100aaa282c2d None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:46.085292 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 103/213] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:46.091982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42126e-7ff0-45d4-a3be-100aaa282c2d None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:46.118676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:46.119912 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:46.132891 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 111/214] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:46.158200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405bd3b-0705-4e2d-a33c-3353a87fe81b None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:46.230224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc42126e-7ff0-45d4-a3be-100aaa282c2d None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:46.270791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405bd3b-0705-4e2d-a33c-3353a87fe81b None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:46.276618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35390f66-c840-499b-9c11-09bd9bc96cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:46.277273 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35390f66-c840-499b-9c11-09bd9bc96cff None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:46.277556 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35390f66-c840-499b-9c11-09bd9bc96cff None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:46.277832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35390f66-c840-499b-9c11-09bd9bc96cff None admin] PATH_INFO: `/v3/users/382eaf096a50409496cf5a5c8d8b5ea5` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:46.296194 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35390f66-c840-499b-9c11-09bd9bc96cff None admin] RBAC: Authorizing `identity:get_user(user_id=382eaf096a50409496cf5a5c8d8b5ea5)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:46.298917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35390f66-c840-499b-9c11-09bd9bc96cff None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:46.301343 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 104/215] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:46 2022] GET /identity/v3/users/382eaf096a50409496cf5a5c8d8b5ea5 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:46.306778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35390f66-c840-499b-9c11-09bd9bc96cff None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:46.313063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008b146-8e67-4992-ad33-7cff85891bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:46.314282 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008b146-8e67-4992-ad33-7cff85891bee None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:46.314590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008b146-8e67-4992-ad33-7cff85891bee None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:46.314867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008b146-8e67-4992-ad33-7cff85891bee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:46.316007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008b146-8e67-4992-ad33-7cff85891bee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:46.317438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008b146-8e67-4992-ad33-7cff85891bee None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:46.323814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 112/216] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:46.324663 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008b146-8e67-4992-ad33-7cff85891bee None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:46.391939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:46.396535 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:46.427835 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35390f66-c840-499b-9c11-09bd9bc96cff None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:46.454379 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008b146-8e67-4992-ad33-7cff85891bee None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:46.471767 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3814bd1-db90-4032-b228-d59c417de355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:46.472759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3814bd1-db90-4032-b228-d59c417de355 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:46.473187 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3814bd1-db90-4032-b228-d59c417de355 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:46.474004 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3814bd1-db90-4032-b228-d59c417de355 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:46.491670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e567eee-fcd3-4255-806b-5a49af622f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:46.492371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e567eee-fcd3-4255-806b-5a49af622f32 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:46.492744 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e567eee-fcd3-4255-806b-5a49af622f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:46.493112 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e567eee-fcd3-4255-806b-5a49af622f32 None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:46.493896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e567eee-fcd3-4255-806b-5a49af622f32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:46.495496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e567eee-fcd3-4255-806b-5a49af622f32 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:46.521115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 113/217] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:39:46 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:39:46.554975 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3814bd1-db90-4032-b228-d59c417de355 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:46.556708 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3814bd1-db90-4032-b228-d59c417de355 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:46.634743 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:46.669269 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 105/218] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 12:39:46.675761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e567eee-fcd3-4255-806b-5a49af622f32 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:46.710451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ee36d-ad15-44a8-a204-2769f665b714 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:46.710821 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ee36d-ad15-44a8-a204-2769f665b714 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:46.711060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ee36d-ad15-44a8-a204-2769f665b714 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:46.712420 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 106/219] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:39:46.717410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e91e5b-8379-49b9-abf4-fa75c6bfc250 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:46.717694 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e91e5b-8379-49b9-abf4-fa75c6bfc250 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:46.717965 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e91e5b-8379-49b9-abf4-fa75c6bfc250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:46.737605 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-75e91e5b-8379-49b9-abf4-fa75c6bfc250 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:46.738920 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-75e91e5b-8379-49b9-abf4-fa75c6bfc250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:46.801735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e567eee-fcd3-4255-806b-5a49af622f32 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:46.818274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:46.824250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 107/220] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:46.840545 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e86f5a-fb4a-4e79-8a31-8bd2458c211d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:46.841258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e86f5a-fb4a-4e79-8a31-8bd2458c211d None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:46.841538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e86f5a-fb4a-4e79-8a31-8bd2458c211d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:46.841807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e86f5a-fb4a-4e79-8a31-8bd2458c211d None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:46.842990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e86f5a-fb4a-4e79-8a31-8bd2458c211d None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:46.844489 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e86f5a-fb4a-4e79-8a31-8bd2458c211d None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:46.847087 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 114/221] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:46 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:46.849725 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e91e5b-8379-49b9-abf4-fa75c6bfc250 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:46.852668 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e86f5a-fb4a-4e79-8a31-8bd2458c211d None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:46.987742 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e91e5b-8379-49b9-abf4-fa75c6bfc250 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:46.988268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e86f5a-fb4a-4e79-8a31-8bd2458c211d None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:46.989542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:47.028666 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c98ea-8b8c-4419-ae2e-e63c2008fd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:47.029343 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c98ea-8b8c-4419-ae2e-e63c2008fd7f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:47.029620 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c98ea-8b8c-4419-ae2e-e63c2008fd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:47.029887 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c98ea-8b8c-4419-ae2e-e63c2008fd7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:47.034836 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647bc00-8b0b-455b-8335-7cf288ce087b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:47.035711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647bc00-8b0b-455b-8335-7cf288ce087b None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:47.036008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647bc00-8b0b-455b-8335-7cf288ce087b None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:47.036388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647bc00-8b0b-455b-8335-7cf288ce087b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:47.037367 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d647bc00-8b0b-455b-8335-7cf288ce087b None admin] RBAC: Adding query filter params (role.id=0f74302099d9402ba2a922f36cff56ef, scope.project.id=efec6052310d4c7289f0ce48ac293400, user.id=382eaf096a50409496cf5a5c8d8b5ea5) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:47.037675 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d647bc00-8b0b-455b-8335-7cf288ce087b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:47.039734 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d647bc00-8b0b-455b-8335-7cf288ce087b None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:47.041696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c98ea-8b8c-4419-ae2e-e63c2008fd7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:47.046216 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 115/222] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:46 2022] GET /identity/v3/role_assignments?user.id=382eaf096a50409496cf5a5c8d8b5ea5&scope.project.id=efec6052310d4c7289f0ce48ac293400&role.id=0f74302099d9402ba2a922f36cff56ef => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:47.047279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04082ef-5b5a-4e14-ab0b-9d8376c95b92 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:47.047645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04082ef-5b5a-4e14-ab0b-9d8376c95b92 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:47.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-dd4c98ea-8b8c-4419-ae2e-e63c2008fd7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:47.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:47.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:47.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:47.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:47.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:47.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:47.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:47.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:47.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:47.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:47.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:47.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:47.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:47.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:47.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:47.049104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:47.049104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:47.049104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:47.049104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:47.049104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:47.049104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:47.049104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:47.049104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:47.049104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:47.049104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:47.049104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:47.049104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:47.049104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:47.049104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:47.049104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:47.049104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:47.050469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:47.050469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:47.050469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:47.050469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:47.050469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:47.050469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:47.050469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:47.050469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:47.050469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:47.050469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:47.050469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:47.050469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:47.051292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04082ef-5b5a-4e14-ab0b-9d8376c95b92 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:47.051362 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 108/223] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:39:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:39:47.051434 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 116/224] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:47.055109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c98ea-8b8c-4419-ae2e-e63c2008fd7f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:47.069248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac5bb5-1c9f-4ac7-959c-2fff071eb999 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:47.070362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac5bb5-1c9f-4ac7-959c-2fff071eb999 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:47.071144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ac5bb5-1c9f-4ac7-959c-2fff071eb999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:47.096164 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-56ac5bb5-1c9f-4ac7-959c-2fff071eb999 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:47.097763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-56ac5bb5-1c9f-4ac7-959c-2fff071eb999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:47.193867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:47.225946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c98ea-8b8c-4419-ae2e-e63c2008fd7f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:47.253768 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 117/225] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:39:47.269007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c87c23-50b3-47e3-a681-743e58b28ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:47.269641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c87c23-50b3-47e3-a681-743e58b28ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:47.269921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c87c23-50b3-47e3-a681-743e58b28ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:47.270193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c87c23-50b3-47e3-a681-743e58b28ee3 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:47.270969 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c87c23-50b3-47e3-a681-743e58b28ee3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:47.271276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c87c23-50b3-47e3-a681-743e58b28ee3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:47.272652 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c87c23-50b3-47e3-a681-743e58b28ee3 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:47.281432 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 109/226] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:47.287628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c87c23-50b3-47e3-a681-743e58b28ee3 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:47.287854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac5bb5-1c9f-4ac7-959c-2fff071eb999 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:47.333630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:47.434674 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c87c23-50b3-47e3-a681-743e58b28ee3 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:47.475308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ac5bb5-1c9f-4ac7-959c-2fff071eb999 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:47.509719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c009b5-ccb9-412b-8b7e-f54ec5dd3e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:47.510312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c009b5-ccb9-412b-8b7e-f54ec5dd3e5d None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:47.510627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c009b5-ccb9-412b-8b7e-f54ec5dd3e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:47.510903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c009b5-ccb9-412b-8b7e-f54ec5dd3e5d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:47.511763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96631c-459c-42e9-83fc-9669b33736d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:47.512419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96631c-459c-42e9-83fc-9669b33736d5 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:47.512702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96631c-459c-42e9-83fc-9669b33736d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:47.512981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd96631c-459c-42e9-83fc-9669b33736d5 None admin] PATH_INFO: `/v3/users/c9e3cde4b4a04a5c9d2f82a577a31030` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:47.514185 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96631c-459c-42e9-83fc-9669b33736d5 None admin] RBAC: Authorizing `identity:get_user(user_id=c9e3cde4b4a04a5c9d2f82a577a31030)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:47.515688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd96631c-459c-42e9-83fc-9669b33736d5 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:47.517880 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 118/227] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:47 2022] GET /identity/v3/users/c9e3cde4b4a04a5c9d2f82a577a31030 => generated 283 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:47.527458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96631c-459c-42e9-83fc-9669b33736d5 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:47.527617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c009b5-ccb9-412b-8b7e-f54ec5dd3e5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:47.533613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-78c009b5-ccb9-412b-8b7e-f54ec5dd3e5d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:39:47.533613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:47.533613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:47.533613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:47.533613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:47.533613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:47.533613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:47.533613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:47.533613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:47.533613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:47.533613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:47.533613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:47.533613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:47.533613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:47.533613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:47.533613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:47.534498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:47.534498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:47.534498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:47.534498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:47.534498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:47.534498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:47.534498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:47.534498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:47.534498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:47.534498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:47.534498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:47.534498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:47.534498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:47.534498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:47.534498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:47.534498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:47.535314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:47.535314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:47.535314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:47.535314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:47.535314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:47.535314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:47.535314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:47.535314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:47.535314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:47.535314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:47.535314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:39:47.535314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:47.535314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 110/228] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:39:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:39:47.540641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c009b5-ccb9-412b-8b7e-f54ec5dd3e5d None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:47.625651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:47.640870 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:47.666565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c009b5-ccb9-412b-8b7e-f54ec5dd3e5d None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:47.711286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aee363-a75e-45fb-895b-4315808aed49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:47.712337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aee363-a75e-45fb-895b-4315808aed49 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:47.712916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aee363-a75e-45fb-895b-4315808aed49 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:47.713195 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aee363-a75e-45fb-895b-4315808aed49 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:47.713964 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aee363-a75e-45fb-895b-4315808aed49 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:47.714274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aee363-a75e-45fb-895b-4315808aed49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:47.715683 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aee363-a75e-45fb-895b-4315808aed49 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:47.730499 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 111/229] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:39:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:47.731726 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aee363-a75e-45fb-895b-4315808aed49 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:47.761332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd96631c-459c-42e9-83fc-9669b33736d5 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:47.824683 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3130ea37-75a5-4bf9-a23b-9bae70311020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:47.825326 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130ea37-75a5-4bf9-a23b-9bae70311020 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:47.825600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130ea37-75a5-4bf9-a23b-9bae70311020 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:47.825867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3130ea37-75a5-4bf9-a23b-9bae70311020 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:47.827951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3130ea37-75a5-4bf9-a23b-9bae70311020 None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:47.829913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3130ea37-75a5-4bf9-a23b-9bae70311020 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:47.834090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 119/230] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:47 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:47.839022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3130ea37-75a5-4bf9-a23b-9bae70311020 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:47.912429 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aee363-a75e-45fb-895b-4315808aed49 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:47.984610 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f1e7f-8179-4898-93c1-faf6931449de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:47.985653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f1e7f-8179-4898-93c1-faf6931449de None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:47.986741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f1e7f-8179-4898-93c1-faf6931449de None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:47.987967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f1e7f-8179-4898-93c1-faf6931449de None admin] PATH_INFO: `/v3/users/c9e3cde4b4a04a5c9d2f82a577a31030` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:47.990854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f1e7f-8179-4898-93c1-faf6931449de None admin] RBAC: Authorizing `identity:get_user(user_id=c9e3cde4b4a04a5c9d2f82a577a31030)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:47.993670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f1e7f-8179-4898-93c1-faf6931449de None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:47.998997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 112/231] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:47 2022] GET /identity/v3/users/c9e3cde4b4a04a5c9d2f82a577a31030 => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:48.005164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f1e7f-8179-4898-93c1-faf6931449de None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:48.008715 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3130ea37-75a5-4bf9-a23b-9bae70311020 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:48.037807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:48.042116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:48.080823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613efd96-bf63-4d16-842a-8daa3575347f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:48.085265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613efd96-bf63-4d16-842a-8daa3575347f None admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:48.085793 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613efd96-bf63-4d16-842a-8daa3575347f None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:48.086268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613efd96-bf63-4d16-842a-8daa3575347f None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400/users/c9e3cde4b4a04a5c9d2f82a577a31030/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:48.089314 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613efd96-bf63-4d16-842a-8daa3575347f None admin] RBAC: Authorizing `identity:create_grant(project_id=efec6052310d4c7289f0ce48ac293400, user_id=c9e3cde4b4a04a5c9d2f82a577a31030, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:48.090992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613efd96-bf63-4d16-842a-8daa3575347f None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:48.117555 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 120/232] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:39:47 2022] PUT /identity/v3/projects/efec6052310d4c7289f0ce48ac293400/users/c9e3cde4b4a04a5c9d2f82a577a31030/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:39:48.176772 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f1e7f-8179-4898-93c1-faf6931449de None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:48.236955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ffc04-cfbe-4ff9-86af-2c569c164589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:48.237640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ffc04-cfbe-4ff9-86af-2c569c164589 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:48.237926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ffc04-cfbe-4ff9-86af-2c569c164589 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:48.238205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ffc04-cfbe-4ff9-86af-2c569c164589 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:48.243432 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ffc04-cfbe-4ff9-86af-2c569c164589 None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:48.244938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ffc04-cfbe-4ff9-86af-2c569c164589 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:48.246817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 113/233] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:48 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:48.253261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613efd96-bf63-4d16-842a-8daa3575347f None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:48.398086 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613efd96-bf63-4d16-842a-8daa3575347f None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:48.446655 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd8efc-a357-4fc3-81c9-66a374e244c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:48.447331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd8efc-a357-4fc3-81c9-66a374e244c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:48.447606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd8efc-a357-4fc3-81c9-66a374e244c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:48.447876 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd8efc-a357-4fc3-81c9-66a374e244c0 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400/users/c9e3cde4b4a04a5c9d2f82a577a31030/roles/79ff845a41e241229433560e2b0d6fe5` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:48.450092 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd8efc-a357-4fc3-81c9-66a374e244c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=efec6052310d4c7289f0ce48ac293400, user_id=c9e3cde4b4a04a5c9d2f82a577a31030, role_id=79ff845a41e241229433560e2b0d6fe5)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:48.451557 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fd8efc-a357-4fc3-81c9-66a374e244c0 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:48.466927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 121/234] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:39:48 2022] PUT /identity/v3/projects/efec6052310d4c7289f0ce48ac293400/users/c9e3cde4b4a04a5c9d2f82a577a31030/roles/79ff845a41e241229433560e2b0d6fe5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:39:49.001087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426536d-c09e-4750-b0ae-aa08e3b4da7b None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:49.001628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426536d-c09e-4750-b0ae-aa08e3b4da7b None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:49.001628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426536d-c09e-4750-b0ae-aa08e3b4da7b None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:49.003294 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 114/235] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:49.008613 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b69f9-9e72-4353-9364-2ba55383d9df None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:49.009127 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b69f9-9e72-4353-9364-2ba55383d9df None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:49.010009 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b69f9-9e72-4353-9364-2ba55383d9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:49.035528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-ec4b69f9-9e72-4353-9364-2ba55383d9df None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:49.037071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ec4b69f9-9e72-4353-9364-2ba55383d9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:49.076062 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:49.123586 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 122/236] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:49.146546 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426536d-c09e-4750-b0ae-aa08e3b4da7b None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:49.257957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:49.274164 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ccdd8-2943-46bf-bb93-c26083df2b59 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:49.274852 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ccdd8-2943-46bf-bb93-c26083df2b59 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:49.275335 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ccdd8-2943-46bf-bb93-c26083df2b59 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:49.277179 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 123/237] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:49.282946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426536d-c09e-4750-b0ae-aa08e3b4da7b None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:49.283287 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b431aa-81be-4017-a213-e9a453c45a67 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:49.283791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b431aa-81be-4017-a213-e9a453c45a67 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:49.284265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b431aa-81be-4017-a213-e9a453c45a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:49.315228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-78b431aa-81be-4017-a213-e9a453c45a67 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:49.316946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-78b431aa-81be-4017-a213-e9a453c45a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:49.349743 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88334fd7-73f7-4bc7-90a5-18462ca2cf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:49.350811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88334fd7-73f7-4bc7-90a5-18462ca2cf27 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:49.351298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88334fd7-73f7-4bc7-90a5-18462ca2cf27 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:49.351772 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88334fd7-73f7-4bc7-90a5-18462ca2cf27 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:49.369936 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88334fd7-73f7-4bc7-90a5-18462ca2cf27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:49.377114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-88334fd7-73f7-4bc7-90a5-18462ca2cf27 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:39:49.377114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:49.377114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:49.377114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:49.377114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:49.377114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:49.377114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:49.377114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:49.377114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:49.377114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:49.377114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:49.377114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:49.377114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:49.377114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:49.377114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:49.377114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:49.377992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:49.377992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:49.377992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:49.377992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:49.377992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:49.377992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:49.377992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:49.377992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:49.377992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:49.377992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:49.377992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:49.377992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:49.377992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:49.377992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:49.377992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:49.377992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:49.378846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:49.378846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:49.378846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:49.378846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:49.378846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:49.378846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:49.378846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:49.378846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:49.378846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:49.378846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:49.378846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:39:49.378846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:49.385253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 115/238] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:49.394244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88334fd7-73f7-4bc7-90a5-18462ca2cf27 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:49.452852 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 124/239] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:39:49.515309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b431aa-81be-4017-a213-e9a453c45a67 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:49.566966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88334fd7-73f7-4bc7-90a5-18462ca2cf27 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:49.615100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24754b4a-3bd1-474a-8ed6-dd8dcdf03bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:49.615718 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24754b4a-3bd1-474a-8ed6-dd8dcdf03bad None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:49.616009 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24754b4a-3bd1-474a-8ed6-dd8dcdf03bad None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:49.616264 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24754b4a-3bd1-474a-8ed6-dd8dcdf03bad None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:49.616974 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24754b4a-3bd1-474a-8ed6-dd8dcdf03bad None admin] RBAC: Adding query filter params (name=reader) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:49.617273 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24754b4a-3bd1-474a-8ed6-dd8dcdf03bad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:49.618821 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24754b4a-3bd1-474a-8ed6-dd8dcdf03bad None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:49.628085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 116/240] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:49.633369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75a3c4-866c-48c9-abdd-6a4c04ffba60 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:49.633369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75a3c4-866c-48c9-abdd-6a4c04ffba60 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:49.633369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75a3c4-866c-48c9-abdd-6a4c04ffba60 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:49.633369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 117/241] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:49.635404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75a3c4-866c-48c9-abdd-6a4c04ffba60 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:49.643571 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:49.657217 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:49.665403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b431aa-81be-4017-a213-e9a453c45a67 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:49.703657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2ee51-c4a4-40ad-8b73-78b312cf121f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:49.704976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2ee51-c4a4-40ad-8b73-78b312cf121f None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:49.705256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2ee51-c4a4-40ad-8b73-78b312cf121f None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:49.705525 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2ee51-c4a4-40ad-8b73-78b312cf121f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:49.707045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2ee51-c4a4-40ad-8b73-78b312cf121f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:49.708643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2ee51-c4a4-40ad-8b73-78b312cf121f None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:49.714812 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 125/242] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:49.714812 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576c5b4-ccd9-43b2-b830-af4cbef08e5f None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:49.714812 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576c5b4-ccd9-43b2-b830-af4cbef08e5f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:49.714812 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576c5b4-ccd9-43b2-b830-af4cbef08e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:49.731207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-5576c5b4-ccd9-43b2-b830-af4cbef08e5f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:49.732383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5576c5b4-ccd9-43b2-b830-af4cbef08e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:49.771532 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75a3c4-866c-48c9-abdd-6a4c04ffba60 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:49.814744 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 126/243] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:49.815764 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576c5b4-ccd9-43b2-b830-af4cbef08e5f None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:49.817093 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37786d-7d09-4817-9bff-ac4f2386255c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:49.818324 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37786d-7d09-4817-9bff-ac4f2386255c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:49.819789 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37786d-7d09-4817-9bff-ac4f2386255c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:49.820306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37786d-7d09-4817-9bff-ac4f2386255c None admin] PATH_INFO: `/v3/users/21bfe799132e4d30b5afd7d822d61afa` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:49.838260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37786d-7d09-4817-9bff-ac4f2386255c None admin] RBAC: Authorizing `identity:get_user(user_id=21bfe799132e4d30b5afd7d822d61afa)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:49.840751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37786d-7d09-4817-9bff-ac4f2386255c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:49.843883 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 118/244] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:49 2022] GET /identity/v3/users/21bfe799132e4d30b5afd7d822d61afa => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:49.851329 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37786d-7d09-4817-9bff-ac4f2386255c None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:49.921264 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:49.945626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:49.956271 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576c5b4-ccd9-43b2-b830-af4cbef08e5f None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:49.962762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37786d-7d09-4817-9bff-ac4f2386255c None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:49.994179 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33688d93-01e8-4dc1-aac5-2fda018e589d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:49.995163 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33688d93-01e8-4dc1-aac5-2fda018e589d None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:49.995665 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33688d93-01e8-4dc1-aac5-2fda018e589d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:49.996131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33688d93-01e8-4dc1-aac5-2fda018e589d None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:49.997002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33688d93-01e8-4dc1-aac5-2fda018e589d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:49.998241 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eac609-5d57-42bb-9cd5-ce39f9ee39d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:49.998909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eac609-5d57-42bb-9cd5-ce39f9ee39d8 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:49.999033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33688d93-01e8-4dc1-aac5-2fda018e589d None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:49.999194 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eac609-5d57-42bb-9cd5-ce39f9ee39d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:49.999449 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eac609-5d57-42bb-9cd5-ce39f9ee39d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:50.010230 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eac609-5d57-42bb-9cd5-ce39f9ee39d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:50.015682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-e2eac609-5d57-42bb-9cd5-ce39f9ee39d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:50.015682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:50.015682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:50.015682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:50.015682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:50.015682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:50.015682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:50.015682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:50.015682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:50.015682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:50.015682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:50.015682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:50.015682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:50.015682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:50.015682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:50.015682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:50.017151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:50.017151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:50.017151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:50.017151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:50.017151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:50.017151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:50.017151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:50.017151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:50.017151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:50.017151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:50.017151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:50.017151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:50.017151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:50.017151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:50.017151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:50.017151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:50.017957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:50.017957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:50.017957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:50.017957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:50.017957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:50.017957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:50.017957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:50.017957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:50.017957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:50.017957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:50.017957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:50.017957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:50.017957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 119/245] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:50.018744 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eac609-5d57-42bb-9cd5-ce39f9ee39d8 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:50.031265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 127/246] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:39:49 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:39:50.039873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33688d93-01e8-4dc1-aac5-2fda018e589d None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:50.145392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eac609-5d57-42bb-9cd5-ce39f9ee39d8 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:50.176203 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:50.192110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a83237-cdef-4210-8252-a8723c2b817d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:50.192764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83237-cdef-4210-8252-a8723c2b817d None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:50.193042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83237-cdef-4210-8252-a8723c2b817d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:50.193315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83237-cdef-4210-8252-a8723c2b817d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:50.256817 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33688d93-01e8-4dc1-aac5-2fda018e589d None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:50.280537 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a83237-cdef-4210-8252-a8723c2b817d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:50.282341 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a83237-cdef-4210-8252-a8723c2b817d None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:50.325705 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba189a1b-e826-43bf-9662-b27967c71133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:50.326774 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba189a1b-e826-43bf-9662-b27967c71133 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:50.327259 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba189a1b-e826-43bf-9662-b27967c71133 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:50.327760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba189a1b-e826-43bf-9662-b27967c71133 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:50.328684 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba189a1b-e826-43bf-9662-b27967c71133 None admin] RBAC: Adding query filter params (name=member) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:50.329776 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba189a1b-e826-43bf-9662-b27967c71133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:50.331515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba189a1b-e826-43bf-9662-b27967c71133 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:50.350401 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 128/247] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:50.366979 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba189a1b-e826-43bf-9662-b27967c71133 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:50.392888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:50.409479 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 120/248] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:39:50.416590 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a83237-cdef-4210-8252-a8723c2b817d None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:50.447889 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:50.528450 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba189a1b-e826-43bf-9662-b27967c71133 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:50.550370 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a83237-cdef-4210-8252-a8723c2b817d None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:50.581289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d73c60-e83d-436b-88b5-93a930de8bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:50.582449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d73c60-e83d-436b-88b5-93a930de8bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:50.583407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d73c60-e83d-436b-88b5-93a930de8bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:50.584083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d73c60-e83d-436b-88b5-93a930de8bd7 None admin] PATH_INFO: `/v3/users/382eaf096a50409496cf5a5c8d8b5ea5` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:50.586005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d73c60-e83d-436b-88b5-93a930de8bd7 None admin] RBAC: Authorizing `identity:get_user(user_id=382eaf096a50409496cf5a5c8d8b5ea5)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:50.588406 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d73c60-e83d-436b-88b5-93a930de8bd7 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:50.590778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 129/249] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:50 2022] GET /identity/v3/users/382eaf096a50409496cf5a5c8d8b5ea5 => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:50.597752 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1347e-9e53-46be-91a4-6e1ddc94ff58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:50.597752 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1347e-9e53-46be-91a4-6e1ddc94ff58 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:50.597752 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1347e-9e53-46be-91a4-6e1ddc94ff58 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:50.597752 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1347e-9e53-46be-91a4-6e1ddc94ff58 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:50.598848 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1347e-9e53-46be-91a4-6e1ddc94ff58 None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:50.599302 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d73c60-e83d-436b-88b5-93a930de8bd7 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:50.601064 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1347e-9e53-46be-91a4-6e1ddc94ff58 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:50.607802 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 121/250] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:50 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:50.610957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1347e-9e53-46be-91a4-6e1ddc94ff58 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:50.719036 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:50.734571 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:50.747507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1347e-9e53-46be-91a4-6e1ddc94ff58 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:50.752709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d73c60-e83d-436b-88b5-93a930de8bd7 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:50.795556 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07247682-f19a-440d-8b1b-f7700e2fdb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:50.798239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07247682-f19a-440d-8b1b-f7700e2fdb85 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:50.798239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07247682-f19a-440d-8b1b-f7700e2fdb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:50.798239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07247682-f19a-440d-8b1b-f7700e2fdb85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:50.798908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07247682-f19a-440d-8b1b-f7700e2fdb85 None admin] RBAC: Adding query filter params (role.id=ebfd821c64dd448cae68d5408ae32993, scope.project.id=efec6052310d4c7289f0ce48ac293400, user.id=21bfe799132e4d30b5afd7d822d61afa) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:50.799247 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07247682-f19a-440d-8b1b-f7700e2fdb85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:50.800927 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07247682-f19a-440d-8b1b-f7700e2fdb85 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:50.801933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39d1a3-6025-49f3-8c60-a7134ea28d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:50.802613 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39d1a3-6025-49f3-8c60-a7134ea28d01 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:50.803166 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39d1a3-6025-49f3-8c60-a7134ea28d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:50.803530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39d1a3-6025-49f3-8c60-a7134ea28d01 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:50.805442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b39d1a3-6025-49f3-8c60-a7134ea28d01 None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:50.807064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b39d1a3-6025-49f3-8c60-a7134ea28d01 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:50.807198 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 122/251] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:50 2022] GET /identity/v3/role_assignments?user.id=21bfe799132e4d30b5afd7d822d61afa&scope.project.id=efec6052310d4c7289f0ce48ac293400&role.id=ebfd821c64dd448cae68d5408ae32993 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:50.808384 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab3cf5-3d6e-41e3-8472-088988a89230 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:50.808665 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab3cf5-3d6e-41e3-8472-088988a89230 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:50.808944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab3cf5-3d6e-41e3-8472-088988a89230 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:50.809344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 130/252] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:50 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:50.810523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 123/253] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:50.815042 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5d0f0-ae49-4a73-b2b7-89de6d17e434 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:50.815333 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5d0f0-ae49-4a73-b2b7-89de6d17e434 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:50.815614 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5d0f0-ae49-4a73-b2b7-89de6d17e434 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:50.817059 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 131/254] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:50.818508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab3cf5-3d6e-41e3-8472-088988a89230 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:50.820105 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0de5b-44b6-4fc5-8b18-9d55a659dff7 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:50.820567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0de5b-44b6-4fc5-8b18-9d55a659dff7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:50.821035 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0de5b-44b6-4fc5-8b18-9d55a659dff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:50.854409 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-fcb0de5b-44b6-4fc5-8b18-9d55a659dff7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:50.855677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-fcb0de5b-44b6-4fc5-8b18-9d55a659dff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:50.960101 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 132/255] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:39:50.962626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3f175-1886-403b-856a-3d193d6e4ca6 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:50.963088 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3f175-1886-403b-856a-3d193d6e4ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:50.963780 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c3f175-1886-403b-856a-3d193d6e4ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:50.998300 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-31c3f175-1886-403b-856a-3d193d6e4ca6 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:51.000425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-31c3f175-1886-403b-856a-3d193d6e4ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:51.039216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab3cf5-3d6e-41e3-8472-088988a89230 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:51.078035 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676c044-3cd4-4fe3-9965-0cd3509da07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:51.078759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676c044-3cd4-4fe3-9965-0cd3509da07d None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:51.080708 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676c044-3cd4-4fe3-9965-0cd3509da07d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:51.081066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4676c044-3cd4-4fe3-9965-0cd3509da07d None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400/users/382eaf096a50409496cf5a5c8d8b5ea5/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:51.083253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676c044-3cd4-4fe3-9965-0cd3509da07d None admin] RBAC: Authorizing `identity:create_grant(project_id=efec6052310d4c7289f0ce48ac293400, user_id=382eaf096a50409496cf5a5c8d8b5ea5, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:51.084794 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4676c044-3cd4-4fe3-9965-0cd3509da07d None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:51.092766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 133/256] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:51.094316 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3f175-1886-403b-856a-3d193d6e4ca6 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:51.120074 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:51.121149 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 124/257] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:39:50 2022] PUT /identity/v3/projects/efec6052310d4c7289f0ce48ac293400/users/382eaf096a50409496cf5a5c8d8b5ea5/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:39:51.136023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676c044-3cd4-4fe3-9965-0cd3509da07d None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:51.158103 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:51.312181 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4676c044-3cd4-4fe3-9965-0cd3509da07d None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:51.334848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c3f175-1886-403b-856a-3d193d6e4ca6 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:51.377480 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798fa8a-aaaa-4ec8-8272-4d660dd4d049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:51.378673 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798fa8a-aaaa-4ec8-8272-4d660dd4d049 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:51.378978 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798fa8a-aaaa-4ec8-8272-4d660dd4d049 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:51.379254 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8798fa8a-aaaa-4ec8-8272-4d660dd4d049 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:51.400848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b745262-5002-4e96-b706-26daf47b7a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:51.401607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b745262-5002-4e96-b706-26daf47b7a49 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:51.401886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b745262-5002-4e96-b706-26daf47b7a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:51.402155 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b745262-5002-4e96-b706-26daf47b7a49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:51.410234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8798fa8a-aaaa-4ec8-8272-4d660dd4d049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:51.415307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-8798fa8a-aaaa-4ec8-8272-4d660dd4d049 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:39:51.415307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:51.415307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:51.415307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:51.415307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:51.415307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:51.415307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:51.415307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:51.415307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:51.415307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:51.415307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:51.415307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:51.415307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:51.415307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:51.415307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:51.415307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:51.416178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:51.416178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:51.416178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:51.416178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:51.416178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:51.416178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:51.416178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:51.416178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:51.416178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:51.416178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:51.416178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:51.416178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:51.416178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:51.416178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:51.416178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:51.416178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:51.416971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:51.416971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:51.416971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:51.416971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:51.416971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:51.416971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:51.416971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:51.416971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:51.416971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:51.416971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:51.416971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:39:51.416971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:51.416971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 125/258] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:39:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:51.422064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b745262-5002-4e96-b706-26daf47b7a49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:51.427418 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798fa8a-aaaa-4ec8-8272-4d660dd4d049 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:51.427611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-5b745262-5002-4e96-b706-26daf47b7a49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:51.427611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:51.427611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:51.427611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:51.427611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:51.427611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:51.427611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:51.427611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:51.427611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:51.427611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:51.427611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:51.427611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:51.427611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:51.427611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:51.427611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:51.427611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:51.428475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:51.428475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:51.428475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:51.428475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:51.428475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:51.428475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:51.428475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:51.428475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:51.428475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:51.428475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:51.428475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:51.428475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:51.428475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:51.428475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:51.428475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:51.428475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:51.429288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:51.429288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:51.429288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:51.429288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:51.429288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:51.429288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:51.429288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:51.429288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:51.429288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:51.429288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:51.429288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:39:51.429288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:39:51.429288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 134/259] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:39:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:51.434725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b745262-5002-4e96-b706-26daf47b7a49 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:51.513672 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:51.556775 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b745262-5002-4e96-b706-26daf47b7a49 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:51.597918 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853e5d1-ac24-4d92-aa84-867eb70acdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:51.598521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853e5d1-ac24-4d92-aa84-867eb70acdb8 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:51.598810 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853e5d1-ac24-4d92-aa84-867eb70acdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:51.599083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853e5d1-ac24-4d92-aa84-867eb70acdb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:51.600537 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:51.600871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853e5d1-ac24-4d92-aa84-867eb70acdb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:51.601197 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853e5d1-ac24-4d92-aa84-867eb70acdb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:51.602678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853e5d1-ac24-4d92-aa84-867eb70acdb8 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:51.617955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 135/260] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:51.619832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853e5d1-ac24-4d92-aa84-867eb70acdb8 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:51.656240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8798fa8a-aaaa-4ec8-8272-4d660dd4d049 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:51.715629 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726c04a-cb6f-43d3-a1c3-0ef4b2cc7438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:51.716258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726c04a-cb6f-43d3-a1c3-0ef4b2cc7438 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:51.716537 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726c04a-cb6f-43d3-a1c3-0ef4b2cc7438 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:51.716809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726c04a-cb6f-43d3-a1c3-0ef4b2cc7438 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:51.717529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726c04a-cb6f-43d3-a1c3-0ef4b2cc7438 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:51.717829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726c04a-cb6f-43d3-a1c3-0ef4b2cc7438 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:51.719230 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726c04a-cb6f-43d3-a1c3-0ef4b2cc7438 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:51.738336 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 126/261] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:39:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:51.742837 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726c04a-cb6f-43d3-a1c3-0ef4b2cc7438 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:51.814823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853e5d1-ac24-4d92-aa84-867eb70acdb8 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:51.853515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726c04a-cb6f-43d3-a1c3-0ef4b2cc7438 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:51.883522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:51.893093 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137bd9a-1630-462a-8e9a-7c4699b5b911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:51.893752 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137bd9a-1630-462a-8e9a-7c4699b5b911 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:51.894031 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137bd9a-1630-462a-8e9a-7c4699b5b911 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:51.894303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d137bd9a-1630-462a-8e9a-7c4699b5b911 None admin] PATH_INFO: `/v3/users/c9e3cde4b4a04a5c9d2f82a577a31030` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:51.895588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137bd9a-1630-462a-8e9a-7c4699b5b911 None admin] RBAC: Authorizing `identity:get_user(user_id=c9e3cde4b4a04a5c9d2f82a577a31030)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:51.897158 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d137bd9a-1630-462a-8e9a-7c4699b5b911 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:51.898718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc040b4b-911e-44c0-8945-eceb89702968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:51.899372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc040b4b-911e-44c0-8945-eceb89702968 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:51.899650 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc040b4b-911e-44c0-8945-eceb89702968 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:51.899923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc040b4b-911e-44c0-8945-eceb89702968 None admin] PATH_INFO: `/v3/users/c9e3cde4b4a04a5c9d2f82a577a31030` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:51.901104 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc040b4b-911e-44c0-8945-eceb89702968 None admin] RBAC: Authorizing `identity:get_user(user_id=c9e3cde4b4a04a5c9d2f82a577a31030)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:51.902548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc040b4b-911e-44c0-8945-eceb89702968 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:51.902659 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 127/262] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:51 2022] GET /identity/v3/users/c9e3cde4b4a04a5c9d2f82a577a31030 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:51.904163 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 136/263] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:51 2022] GET /identity/v3/users/c9e3cde4b4a04a5c9d2f82a577a31030 => generated 283 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:51.904649 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137bd9a-1630-462a-8e9a-7c4699b5b911 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:51.910224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc040b4b-911e-44c0-8945-eceb89702968 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:51.917266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:52.078773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d137bd9a-1630-462a-8e9a-7c4699b5b911 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:52.100876 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc040b4b-911e-44c0-8945-eceb89702968 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:52.114949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b46d6-66c8-4ac5-99ed-66ea3da36ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:52.115553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b46d6-66c8-4ac5-99ed-66ea3da36ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:52.115947 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b46d6-66c8-4ac5-99ed-66ea3da36ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:52.116366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b46d6-66c8-4ac5-99ed-66ea3da36ca9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:52.155196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0d8a6-121e-422c-8bef-32c5b144dc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:52.155840 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0d8a6-121e-422c-8bef-32c5b144dc54 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:52.156213 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0d8a6-121e-422c-8bef-32c5b144dc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:52.156526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0d8a6-121e-422c-8bef-32c5b144dc54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:52.191046 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b46d6-66c8-4ac5-99ed-66ea3da36ca9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:52.192729 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b46d6-66c8-4ac5-99ed-66ea3da36ca9 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:52.195603 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:52.229436 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:52.237152 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0d8a6-121e-422c-8bef-32c5b144dc54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:52.238915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0d8a6-121e-422c-8bef-32c5b144dc54 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:52.305418 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 128/264] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 12:39:52.311527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b46d6-66c8-4ac5-99ed-66ea3da36ca9 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:52.341194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 137/265] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 12:39:52.356024 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0d8a6-121e-422c-8bef-32c5b144dc54 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:52.450845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b46d6-66c8-4ac5-99ed-66ea3da36ca9 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:52.453357 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:52.506026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb8ef6-e253-476b-874f-9f0ebd405600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:52.506806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb8ef6-e253-476b-874f-9f0ebd405600 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:52.507105 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb8ef6-e253-476b-874f-9f0ebd405600 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:52.507459 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb8ef6-e253-476b-874f-9f0ebd405600 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:52.510696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb8ef6-e253-476b-874f-9f0ebd405600 None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:52.512253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb8ef6-e253-476b-874f-9f0ebd405600 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:52.514285 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 129/266] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:52 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:52.520669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb8ef6-e253-476b-874f-9f0ebd405600 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:52.538390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:52.547482 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0d8a6-121e-422c-8bef-32c5b144dc54 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:52.609380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e6c29-3f7c-4506-81be-6e7909ac8015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:52.610043 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e6c29-3f7c-4506-81be-6e7909ac8015 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:52.610905 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e6c29-3f7c-4506-81be-6e7909ac8015 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:52.611177 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e6c29-3f7c-4506-81be-6e7909ac8015 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:52.615350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e6c29-3f7c-4506-81be-6e7909ac8015 None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:52.616992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e6c29-3f7c-4506-81be-6e7909ac8015 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:52.619031 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 138/267] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:52 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:52.628181 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e6c29-3f7c-4506-81be-6e7909ac8015 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:52.699894 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb8ef6-e253-476b-874f-9f0ebd405600 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:52.774131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070853d7-658b-4a75-a277-31b6bbe67a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:52.774796 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070853d7-658b-4a75-a277-31b6bbe67a19 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:52.775074 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070853d7-658b-4a75-a277-31b6bbe67a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:52.775341 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070853d7-658b-4a75-a277-31b6bbe67a19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:52.776253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070853d7-658b-4a75-a277-31b6bbe67a19 None admin] RBAC: Adding query filter params (role.id=79ff845a41e241229433560e2b0d6fe5, scope.project.id=efec6052310d4c7289f0ce48ac293400, user.id=c9e3cde4b4a04a5c9d2f82a577a31030) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:52.776563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070853d7-658b-4a75-a277-31b6bbe67a19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:52.777996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070853d7-658b-4a75-a277-31b6bbe67a19 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:52.784505 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 130/268] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:52 2022] GET /identity/v3/role_assignments?user.id=c9e3cde4b4a04a5c9d2f82a577a31030&scope.project.id=efec6052310d4c7289f0ce48ac293400&role.id=79ff845a41e241229433560e2b0d6fe5 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:52.838356 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e6c29-3f7c-4506-81be-6e7909ac8015 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:52.903414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d687a-ef70-4da6-a293-cffbbbff5b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:52.904041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d687a-ef70-4da6-a293-cffbbbff5b39 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:52.904320 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d687a-ef70-4da6-a293-cffbbbff5b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:52.904588 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d687a-ef70-4da6-a293-cffbbbff5b39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:52.905388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d687a-ef70-4da6-a293-cffbbbff5b39 None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=efec6052310d4c7289f0ce48ac293400, user.id=c9e3cde4b4a04a5c9d2f82a577a31030) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:52.905693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d687a-ef70-4da6-a293-cffbbbff5b39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:52.907232 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40d687a-ef70-4da6-a293-cffbbbff5b39 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:52.916385 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 139/269] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:52 2022] GET /identity/v3/role_assignments?user.id=c9e3cde4b4a04a5c9d2f82a577a31030&scope.project.id=efec6052310d4c7289f0ce48ac293400&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 608 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:53.267374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632d309-b6ed-4fbc-9731-2237e9c9a658 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:53.267911 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632d309-b6ed-4fbc-9731-2237e9c9a658 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:53.267911 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632d309-b6ed-4fbc-9731-2237e9c9a658 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:53.269554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 131/270] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:53.275120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9efb09-5840-458a-8bfd-a07ebb56c356 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:53.275575 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9efb09-5840-458a-8bfd-a07ebb56c356 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:53.275684 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9efb09-5840-458a-8bfd-a07ebb56c356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:53.296571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-6c9efb09-5840-458a-8bfd-a07ebb56c356 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:53.298357 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-6c9efb09-5840-458a-8bfd-a07ebb56c356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:53.323999 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:53.374346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 140/271] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:39:53.411229 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632d309-b6ed-4fbc-9731-2237e9c9a658 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:53.460102 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:53.486944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836248cd-8496-4c3e-bacc-2f72126b490d None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:53.487256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836248cd-8496-4c3e-bacc-2f72126b490d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:53.487830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836248cd-8496-4c3e-bacc-2f72126b490d None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:53.494717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 141/272] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:53.495910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5329a-2d7f-467e-b2d5-b1268c5c301b None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:53.496197 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5329a-2d7f-467e-b2d5-b1268c5c301b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:53.496472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5329a-2d7f-467e-b2d5-b1268c5c301b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:53.522330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-00c5329a-2d7f-467e-b2d5-b1268c5c301b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:53.523882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-00c5329a-2d7f-467e-b2d5-b1268c5c301b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:53.559533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632d309-b6ed-4fbc-9731-2237e9c9a658 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:53.600185 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2d653-9be0-4270-8225-fcf0dfbbfe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:53.600834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2d653-9be0-4270-8225-fcf0dfbbfe83 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:53.601121 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2d653-9be0-4270-8225-fcf0dfbbfe83 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:53.601504 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2d653-9be0-4270-8225-fcf0dfbbfe83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:53.602793 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2d653-9be0-4270-8225-fcf0dfbbfe83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:53.604214 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2d653-9be0-4270-8225-fcf0dfbbfe83 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:53.606146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 132/273] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:53.612024 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 142/274] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:53.619916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2d653-9be0-4270-8225-fcf0dfbbfe83 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:53.634586 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5329a-2d7f-467e-b2d5-b1268c5c301b None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:53.726863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2d653-9be0-4270-8225-fcf0dfbbfe83 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:53.729376 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:53.773421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b814d3b-6ef5-4b24-a1e8-073d232abebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:53.774794 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b814d3b-6ef5-4b24-a1e8-073d232abebf None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:53.776232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b814d3b-6ef5-4b24-a1e8-073d232abebf None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:53.776232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b814d3b-6ef5-4b24-a1e8-073d232abebf None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:53.777249 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b814d3b-6ef5-4b24-a1e8-073d232abebf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:53.779896 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b814d3b-6ef5-4b24-a1e8-073d232abebf None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:53.807904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:53.825951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 133/275] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:39:53 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:39:53.834952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5329a-2d7f-467e-b2d5-b1268c5c301b None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:53.844340 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86110e-1aed-4584-8656-cc75e14b56e6 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:53.844751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86110e-1aed-4584-8656-cc75e14b56e6 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:53.845035 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86110e-1aed-4584-8656-cc75e14b56e6 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:53.851193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 134/276] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:53.862063 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062328b-9c31-4a4d-9cf4-e4da546fedbc None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:53.862355 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062328b-9c31-4a4d-9cf4-e4da546fedbc None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:53.862710 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062328b-9c31-4a4d-9cf4-e4da546fedbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:53.897288 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-2062328b-9c31-4a4d-9cf4-e4da546fedbc None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:53.901475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2062328b-9c31-4a4d-9cf4-e4da546fedbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:53.932187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1c59e-ac3e-4770-8409-a3696646df9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:53.932911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1c59e-ac3e-4770-8409-a3696646df9c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:53.933197 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1c59e-ac3e-4770-8409-a3696646df9c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:53.933473 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1c59e-ac3e-4770-8409-a3696646df9c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:53.957273 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1c59e-ac3e-4770-8409-a3696646df9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:53.965759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-79b1c59e-ac3e-4770-8409-a3696646df9c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:39:53.965759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:53.965759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:53.965759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:53.965759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:53.965759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:53.965759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:53.965759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:53.965759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:53.965759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:53.965759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:53.965759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:53.965759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:53.965759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:53.965759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:53.965759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:53.966967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:53.966967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:53.966967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:53.966967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:53.966967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:53.966967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:53.966967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:53.966967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:53.966967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:53.966967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:53.966967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:53.966967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:53.966967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:53.966967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:53.966967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:53.966967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:53.967858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:53.967858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:53.967858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:53.967858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:53.967858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:53.967858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:53.967858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:53.967858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:53.967858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:53.967858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:53.967858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:39:53.967858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:39:53.969069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 143/277] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:53.975704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1c59e-ac3e-4770-8409-a3696646df9c None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:54.041450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 135/278] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:54.065572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062328b-9c31-4a4d-9cf4-e4da546fedbc None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:54.100547 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1c59e-ac3e-4770-8409-a3696646df9c None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:54.142855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4ff7c-2baf-4115-b610-2be760aa4400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:54.143480 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4ff7c-2baf-4115-b610-2be760aa4400 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:54.143750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4ff7c-2baf-4115-b610-2be760aa4400 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:54.144020 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4ff7c-2baf-4115-b610-2be760aa4400 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:54.144740 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f4ff7c-2baf-4115-b610-2be760aa4400 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:54.145039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f4ff7c-2baf-4115-b610-2be760aa4400 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:54.147374 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f4ff7c-2baf-4115-b610-2be760aa4400 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:54.158723 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 144/279] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:54.163970 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4ff7c-2baf-4115-b610-2be760aa4400 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:54.169892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:54.181408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:54.186433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062328b-9c31-4a4d-9cf4-e4da546fedbc None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:54.234208 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512396c9-e952-4726-9807-2b657506711a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:54.234888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512396c9-e952-4726-9807-2b657506711a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:54.235198 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512396c9-e952-4726-9807-2b657506711a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:54.235464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512396c9-e952-4726-9807-2b657506711a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:54.247624 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512396c9-e952-4726-9807-2b657506711a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:54.253071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-512396c9-e952-4726-9807-2b657506711a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:54.253071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:54.253071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:54.253071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:54.253071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:54.253071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:54.253071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:54.253071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:54.253071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:54.253071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:54.253071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:54.253071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:54.253071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:54.253071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:54.253071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:54.253071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:54.253959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:54.253959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:54.253959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:54.253959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:54.253959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:54.253959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:54.253959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:54.253959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:54.253959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:54.253959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:54.253959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:54.253959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:54.253959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:54.253959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:54.253959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:54.253959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:54.254817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:54.254817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:54.254817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:54.254817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:54.254817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:54.254817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:54.254817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:54.254817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:54.254817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:54.254817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:54.254817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:54.254817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:54.254817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 136/280] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:39:54.255978 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444915a1-aecc-4dad-b3ea-09bc23595aca None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:54.255978 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444915a1-aecc-4dad-b3ea-09bc23595aca None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:54.256131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444915a1-aecc-4dad-b3ea-09bc23595aca None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:54.257653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 137/281] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:54.260303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444915a1-aecc-4dad-b3ea-09bc23595aca None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:54.291863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4ff7c-2baf-4115-b610-2be760aa4400 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:54.330410 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2482f34-36cf-4f97-a599-d04e636a129e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:54.331515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2482f34-36cf-4f97-a599-d04e636a129e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:54.331805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2482f34-36cf-4f97-a599-d04e636a129e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:54.332082 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2482f34-36cf-4f97-a599-d04e636a129e None admin] PATH_INFO: `/v3/users/21bfe799132e4d30b5afd7d822d61afa` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:54.333531 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2482f34-36cf-4f97-a599-d04e636a129e None admin] RBAC: Authorizing `identity:get_user(user_id=21bfe799132e4d30b5afd7d822d61afa)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:54.335484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2482f34-36cf-4f97-a599-d04e636a129e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:54.337627 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 145/282] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:54 2022] GET /identity/v3/users/21bfe799132e4d30b5afd7d822d61afa => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:54.339400 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a62b51-ba46-4908-a50e-22aa77d58c98 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:54.339807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a62b51-ba46-4908-a50e-22aa77d58c98 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:54.340090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a62b51-ba46-4908-a50e-22aa77d58c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:54.362093 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-55a62b51-ba46-4908-a50e-22aa77d58c98 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:54.363722 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-55a62b51-ba46-4908-a50e-22aa77d58c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:54.389439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444915a1-aecc-4dad-b3ea-09bc23595aca None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:54.433733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e05fd6-cfb1-450e-a61d-1c493da8df51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:54.435004 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e05fd6-cfb1-450e-a61d-1c493da8df51 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:54.435357 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e05fd6-cfb1-450e-a61d-1c493da8df51 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:54.435693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e05fd6-cfb1-450e-a61d-1c493da8df51 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:54.436783 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e05fd6-cfb1-450e-a61d-1c493da8df51 None admin] RBAC: Adding query filter params (name=member) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:54.437153 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e05fd6-cfb1-450e-a61d-1c493da8df51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:54.439832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e05fd6-cfb1-450e-a61d-1c493da8df51 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:54.443062 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 146/283] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:54.444125 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a62b51-ba46-4908-a50e-22aa77d58c98 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:54.450619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 138/284] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:54.456081 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e05fd6-cfb1-450e-a61d-1c493da8df51 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:54.457188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:54.475360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:54.575170 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a62b51-ba46-4908-a50e-22aa77d58c98 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:54.594536 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e05fd6-cfb1-450e-a61d-1c493da8df51 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:54.612186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a331a-f476-4966-93a2-ffd319490a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:54.612838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a331a-f476-4966-93a2-ffd319490a65 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:54.613188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a331a-f476-4966-93a2-ffd319490a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:54.613392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a331a-f476-4966-93a2-ffd319490a65 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:54.614685 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a331a-f476-4966-93a2-ffd319490a65 None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:54.616051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a331a-f476-4966-93a2-ffd319490a65 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:54.617920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 147/285] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:54 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:54.623286 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a331a-f476-4966-93a2-ffd319490a65 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:54.640622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a23afa-845e-40c6-9e51-874cb0149b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:54.641936 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a23afa-845e-40c6-9e51-874cb0149b30 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:54.642910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a23afa-845e-40c6-9e51-874cb0149b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:54.643656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a23afa-845e-40c6-9e51-874cb0149b30 None admin] PATH_INFO: `/v3/users/c99130233d9c45daace4c15f47bba1ba` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:54.660873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a23afa-845e-40c6-9e51-874cb0149b30 None admin] RBAC: Authorizing `identity:get_user(user_id=c99130233d9c45daace4c15f47bba1ba)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:54.662628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a23afa-845e-40c6-9e51-874cb0149b30 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:54.664906 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 139/286] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:54 2022] GET /identity/v3/users/c99130233d9c45daace4c15f47bba1ba => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:54.665929 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a23afa-845e-40c6-9e51-874cb0149b30 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:54.708339 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:54.733985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a331a-f476-4966-93a2-ffd319490a65 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:54.739214 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:54.770596 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b849a0-ea79-4246-b1e0-85b1733e41a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:54.771253 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b849a0-ea79-4246-b1e0-85b1733e41a0 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:54.771580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b849a0-ea79-4246-b1e0-85b1733e41a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:54.771856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b849a0-ea79-4246-b1e0-85b1733e41a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:54.772836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a23afa-845e-40c6-9e51-874cb0149b30 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:54.785926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b849a0-ea79-4246-b1e0-85b1733e41a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:54.793041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-e1b849a0-ea79-4246-b1e0-85b1733e41a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:54.793041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:54.793041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:54.793041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:54.793041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:54.793041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:54.793041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:54.793041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:54.793041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:54.793041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:54.793041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:54.793041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:54.793041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:54.793041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:54.793041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:54.793041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:54.793914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:54.793914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:54.793914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:54.793914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:54.793914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:54.793914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:54.793914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:54.793914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:54.793914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:54.793914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:54.793914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:54.793914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:54.793914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:54.793914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:54.793914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:54.793914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:54.794760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:54.794760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:54.794760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:54.794760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:54.794760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:54.794760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:54.794760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:54.794760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:54.794760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:54.794760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:54.794760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:54.794760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:39:54.794760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 148/287] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:54.795595 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b849a0-ea79-4246-b1e0-85b1733e41a0 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:54.814243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213f46a-fe9f-4935-ae9b-94b7d82ace43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:54.815306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213f46a-fe9f-4935-ae9b-94b7d82ace43 None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:54.815849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213f46a-fe9f-4935-ae9b-94b7d82ace43 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:54.816309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5213f46a-fe9f-4935-ae9b-94b7d82ace43 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400/users/21bfe799132e4d30b5afd7d822d61afa/roles/ebfd821c64dd448cae68d5408ae32993` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:54.818608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213f46a-fe9f-4935-ae9b-94b7d82ace43 None admin] RBAC: Authorizing `identity:create_grant(project_id=efec6052310d4c7289f0ce48ac293400, user_id=21bfe799132e4d30b5afd7d822d61afa, role_id=ebfd821c64dd448cae68d5408ae32993)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:54.820298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5213f46a-fe9f-4935-ae9b-94b7d82ace43 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:54.836484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 140/288] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:39:54 2022] PUT /identity/v3/projects/efec6052310d4c7289f0ce48ac293400/users/21bfe799132e4d30b5afd7d822d61afa/roles/ebfd821c64dd448cae68d5408ae32993 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:39:54.838719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213f46a-fe9f-4935-ae9b-94b7d82ace43 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:54.967342 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5213f46a-fe9f-4935-ae9b-94b7d82ace43 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:54.997447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b849a0-ea79-4246-b1e0-85b1733e41a0 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:55.005302 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:55.020013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925a9df-2c4e-4b19-b685-e1ddc9a05f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:55.020013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925a9df-2c4e-4b19-b685-e1ddc9a05f97 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:55.020280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925a9df-2c4e-4b19-b685-e1ddc9a05f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:55.020564 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925a9df-2c4e-4b19-b685-e1ddc9a05f97 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:55.022128 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925a9df-2c4e-4b19-b685-e1ddc9a05f97 None admin] RBAC: Adding query filter params (name=member) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:55.022128 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925a9df-2c4e-4b19-b685-e1ddc9a05f97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:55.022943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925a9df-2c4e-4b19-b685-e1ddc9a05f97 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:55.045305 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 141/289] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:55.049397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925a9df-2c4e-4b19-b685-e1ddc9a05f97 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:55.058774 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2734e6-cb65-4b5f-b855-624d309965e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:55.059459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2734e6-cb65-4b5f-b855-624d309965e3 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:55.059988 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2734e6-cb65-4b5f-b855-624d309965e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:55.060274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2734e6-cb65-4b5f-b855-624d309965e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:55.061135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2734e6-cb65-4b5f-b855-624d309965e3 None admin] RBAC: Adding query filter params (role.id=0f74302099d9402ba2a922f36cff56ef, user.id=c99130233d9c45daace4c15f47bba1ba, scope.system=all) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:55.061448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2734e6-cb65-4b5f-b855-624d309965e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:55.062989 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2734e6-cb65-4b5f-b855-624d309965e3 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:55.066211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:55.069940 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 149/290] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Aug 3 12:39:54 2022] GET /identity/v3/role_assignments?user.id=c99130233d9c45daace4c15f47bba1ba&scope.system=all&role.id=0f74302099d9402ba2a922f36cff56ef => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:55.198584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925a9df-2c4e-4b19-b685-e1ddc9a05f97 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:55.241083 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149ac42-54b1-4442-a4ac-ee9b9cebcd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:55.241741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149ac42-54b1-4442-a4ac-ee9b9cebcd67 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:55.242075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149ac42-54b1-4442-a4ac-ee9b9cebcd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:55.242355 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149ac42-54b1-4442-a4ac-ee9b9cebcd67 None admin] PATH_INFO: `/v3/users/382eaf096a50409496cf5a5c8d8b5ea5` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:55.243606 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149ac42-54b1-4442-a4ac-ee9b9cebcd67 None admin] RBAC: Authorizing `identity:get_user(user_id=382eaf096a50409496cf5a5c8d8b5ea5)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:55.245138 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149ac42-54b1-4442-a4ac-ee9b9cebcd67 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:55.247023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 142/291] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:55 2022] GET /identity/v3/users/382eaf096a50409496cf5a5c8d8b5ea5 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:55.252310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2734e6-cb65-4b5f-b855-624d309965e3 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:55.371545 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2734e6-cb65-4b5f-b855-624d309965e3 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:55.415316 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e8386-98ec-4eab-a3ff-699510a05314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:55.415949 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e8386-98ec-4eab-a3ff-699510a05314 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:55.416228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e8386-98ec-4eab-a3ff-699510a05314 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:55.416501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e8386-98ec-4eab-a3ff-699510a05314 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:55.488369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:55.491383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e8386-98ec-4eab-a3ff-699510a05314 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:55.493016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e8386-98ec-4eab-a3ff-699510a05314 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:55.593493 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 150/292] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:39:55.599394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149ac42-54b1-4442-a4ac-ee9b9cebcd67 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:55.688103 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:55.749202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149ac42-54b1-4442-a4ac-ee9b9cebcd67 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:55.799207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee56170-be38-4ba5-b576-2ebbd29de0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:55.799867 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee56170-be38-4ba5-b576-2ebbd29de0cd None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:55.800152 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee56170-be38-4ba5-b576-2ebbd29de0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:55.800432 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee56170-be38-4ba5-b576-2ebbd29de0cd None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:55.801629 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee56170-be38-4ba5-b576-2ebbd29de0cd None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:55.803085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee56170-be38-4ba5-b576-2ebbd29de0cd None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:55.804748 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 143/293] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:55 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:55.810869 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e8386-98ec-4eab-a3ff-699510a05314 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:55.999573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:56.003580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e8386-98ec-4eab-a3ff-699510a05314 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:56.052763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea13fb2-072f-4abc-86b3-568a6754cbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:56.053386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea13fb2-072f-4abc-86b3-568a6754cbe4 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:56.053666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea13fb2-072f-4abc-86b3-568a6754cbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:56.053939 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea13fb2-072f-4abc-86b3-568a6754cbe4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:56.054758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea13fb2-072f-4abc-86b3-568a6754cbe4 None admin] RBAC: Adding query filter params (role.id=0f74302099d9402ba2a922f36cff56ef, scope.project.id=efec6052310d4c7289f0ce48ac293400, user.id=382eaf096a50409496cf5a5c8d8b5ea5) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:56.055069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea13fb2-072f-4abc-86b3-568a6754cbe4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:56.056487 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea13fb2-072f-4abc-86b3-568a6754cbe4 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:56.065064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 151/294] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:55 2022] GET /identity/v3/role_assignments?user.id=382eaf096a50409496cf5a5c8d8b5ea5&scope.project.id=efec6052310d4c7289f0ce48ac293400&role.id=0f74302099d9402ba2a922f36cff56ef => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:56.465587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72528e-7eaf-41ea-8138-28c4eabc2cfb None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:56.465867 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72528e-7eaf-41ea-8138-28c4eabc2cfb None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:56.466311 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a72528e-7eaf-41ea-8138-28c4eabc2cfb None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:56.468088 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 144/295] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:56.474722 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22854a2a-ff7b-4c25-8f1c-e944054107d0 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:56.475561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22854a2a-ff7b-4c25-8f1c-e944054107d0 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:56.476048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22854a2a-ff7b-4c25-8f1c-e944054107d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:56.496302 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-22854a2a-ff7b-4c25-8f1c-e944054107d0 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:56.498362 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99337e-8e22-40e8-980a-d102652f0ce3 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:56.499677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-22854a2a-ff7b-4c25-8f1c-e944054107d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:56.499955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99337e-8e22-40e8-980a-d102652f0ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:56.500463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99337e-8e22-40e8-980a-d102652f0ce3 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:56.502316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 145/296] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:39:56.507192 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14a584-84c3-4b80-998d-25f1ef7fce0c None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:56.507710 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14a584-84c3-4b80-998d-25f1ef7fce0c None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:56.508190 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f14a584-84c3-4b80-998d-25f1ef7fce0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:56.529360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-1f14a584-84c3-4b80-998d-25f1ef7fce0c None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:56.530685 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1f14a584-84c3-4b80-998d-25f1ef7fce0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:56.586939 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 152/297] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:56.609418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22854a2a-ff7b-4c25-8f1c-e944054107d0 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:56.610672 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 146/298] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:39:56.633578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14a584-84c3-4b80-998d-25f1ef7fce0c None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:56.696434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:56.709417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:56.717110 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22854a2a-ff7b-4c25-8f1c-e944054107d0 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:56.743246 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f14a584-84c3-4b80-998d-25f1ef7fce0c None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:56.755474 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3bc2f1-dc76-4616-a498-e9bfe71d3abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:56.756432 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bc2f1-dc76-4616-a498-e9bfe71d3abb None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:56.756905 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bc2f1-dc76-4616-a498-e9bfe71d3abb None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:56.757367 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bc2f1-dc76-4616-a498-e9bfe71d3abb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:56.758879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3bc2f1-dc76-4616-a498-e9bfe71d3abb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:56.760527 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3bc2f1-dc76-4616-a498-e9bfe71d3abb None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:56.762329 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 153/299] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:56.768467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3bc2f1-dc76-4616-a498-e9bfe71d3abb None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:56.779460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bb8a2-fa18-4b54-800d-eaeade7126d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:56.780110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bb8a2-fa18-4b54-800d-eaeade7126d4 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:56.780439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bb8a2-fa18-4b54-800d-eaeade7126d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:56.780771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bb8a2-fa18-4b54-800d-eaeade7126d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:56.792641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3bb8a2-fa18-4b54-800d-eaeade7126d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:56.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-5d3bb8a2-fa18-4b54-800d-eaeade7126d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:39:56.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:56.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:56.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:56.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:56.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:56.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:56.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:56.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:56.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:56.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:56.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:56.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:56.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:56.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:56.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:56.799091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:56.799091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:56.799091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:56.799091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:56.799091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:56.799091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:56.799091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:56.799091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:56.799091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:56.799091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:56.799091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:56.799091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:56.799091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:56.799091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:56.799091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:56.799091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:56.799892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:56.799892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:56.799892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:56.799892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:56.799892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:56.799892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:56.799892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:56.799892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:56.799892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:56.799892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:56.799892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:39:56.799892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:56.799892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 147/300] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:39:56.804923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bb8a2-fa18-4b54-800d-eaeade7126d4 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:56.878933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3bc2f1-dc76-4616-a498-e9bfe71d3abb None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:56.913128 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bb8a2-fa18-4b54-800d-eaeade7126d4 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:56.916206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbcdbb-7f71-48eb-a90c-6766e4d40530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:56.916931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbcdbb-7f71-48eb-a90c-6766e4d40530 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:56.917306 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbcdbb-7f71-48eb-a90c-6766e4d40530 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:56.917672 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbcdbb-7f71-48eb-a90c-6766e4d40530 None admin] PATH_INFO: `/v3/groups` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:56.918437 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdbcdbb-7f71-48eb-a90c-6766e4d40530 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:56.919992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdbcdbb-7f71-48eb-a90c-6766e4d40530 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:56.931166 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:56.954717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 154/301] 10.222.0.65 () {60 vars in 1134 bytes} [Wed Aug 3 12:39:56 2022] POST /identity/v3/groups => generated 229 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:39:56.956595 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb606c26-91af-4ac0-80ea-e774c28c6d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:56.957304 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb606c26-91af-4ac0-80ea-e774c28c6d2d None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:56.957585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb606c26-91af-4ac0-80ea-e774c28c6d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:56.957904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb606c26-91af-4ac0-80ea-e774c28c6d2d None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:56.958689 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb606c26-91af-4ac0-80ea-e774c28c6d2d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:56.959039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb606c26-91af-4ac0-80ea-e774c28c6d2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:56.960536 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb606c26-91af-4ac0-80ea-e774c28c6d2d None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:56.971894 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 148/302] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:56.977621 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbcdbb-7f71-48eb-a90c-6766e4d40530 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:57.006892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:57.097638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbcdbb-7f71-48eb-a90c-6766e4d40530 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:57.135637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae3f5a-c937-43ae-a123-29091b3c965b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:57.136303 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae3f5a-c937-43ae-a123-29091b3c965b None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:57.136580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae3f5a-c937-43ae-a123-29091b3c965b None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:57.136852 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae3f5a-c937-43ae-a123-29091b3c965b None admin] PATH_INFO: `/v3/users/27215ee848b64056b9ddb890ad6062bf` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:57.151834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae3f5a-c937-43ae-a123-29091b3c965b None admin] RBAC: Authorizing `identity:get_user(user_id=27215ee848b64056b9ddb890ad6062bf)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:57.153296 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eae3f5a-c937-43ae-a123-29091b3c965b None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:57.155205 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 155/303] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:56 2022] GET /identity/v3/users/27215ee848b64056b9ddb890ad6062bf => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:57.161012 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb606c26-91af-4ac0-80ea-e774c28c6d2d None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:57.272762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb606c26-91af-4ac0-80ea-e774c28c6d2d None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:57.292016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa106f-406e-4c21-946c-baddf3499f62 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:57.292531 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa106f-406e-4c21-946c-baddf3499f62 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:57.292807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa106f-406e-4c21-946c-baddf3499f62 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:57.294301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 156/304] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:57.299954 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8c9a2-da92-40a0-a23b-301c4a2b54d1 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:57.300480 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8c9a2-da92-40a0-a23b-301c4a2b54d1 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:57.300958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa8c9a2-da92-40a0-a23b-301c4a2b54d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:57.313743 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c08b40-6b50-46b2-b44b-55f68ce2c456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:57.314378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c08b40-6b50-46b2-b44b-55f68ce2c456 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:57.314758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c08b40-6b50-46b2-b44b-55f68ce2c456 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:57.315031 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c08b40-6b50-46b2-b44b-55f68ce2c456 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:57.316085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c08b40-6b50-46b2-b44b-55f68ce2c456 None admin] RBAC: Adding query filter params (role.id=ebfd821c64dd448cae68d5408ae32993, user.id=27215ee848b64056b9ddb890ad6062bf, scope.system=all) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:57.316085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c08b40-6b50-46b2-b44b-55f68ce2c456 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:57.318199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c08b40-6b50-46b2-b44b-55f68ce2c456 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:57.323361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-daa8c9a2-da92-40a0-a23b-301c4a2b54d1 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:57.324259 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 149/305] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Aug 3 12:39:57 2022] GET /identity/v3/role_assignments?user.id=27215ee848b64056b9ddb890ad6062bf&scope.system=all&role.id=ebfd821c64dd448cae68d5408ae32993 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:57.325108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-daa8c9a2-da92-40a0-a23b-301c4a2b54d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:57.351831 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b8802-dbb4-423c-8246-6d367223c662 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:57.352152 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b8802-dbb4-423c-8246-6d367223c662 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:57.352369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b8802-dbb4-423c-8246-6d367223c662 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:57.354071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 150/306] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:39:57.359596 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b8e1c-e1da-4f70-9c56-9b1fc98244b9 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:57.360552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b8e1c-e1da-4f70-9c56-9b1fc98244b9 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:57.360930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b8e1c-e1da-4f70-9c56-9b1fc98244b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:57.381315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-263b8e1c-e1da-4f70-9c56-9b1fc98244b9 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:57.382768 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-263b8e1c-e1da-4f70-9c56-9b1fc98244b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:57.412049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 157/307] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:39:57.428513 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:57.436454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8c9a2-da92-40a0-a23b-301c4a2b54d1 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:57.451486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:57.470267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 151/308] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:39:57.493142 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b8e1c-e1da-4f70-9c56-9b1fc98244b9 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:57.552614 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa8c9a2-da92-40a0-a23b-301c4a2b54d1 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:57.597878 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e5cb6-4bb5-4e5a-ae27-8bc53f6a312c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:57.598804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e5cb6-4bb5-4e5a-ae27-8bc53f6a312c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:57.599086 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e5cb6-4bb5-4e5a-ae27-8bc53f6a312c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:57.599782 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e5cb6-4bb5-4e5a-ae27-8bc53f6a312c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:57.612672 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b8e1c-e1da-4f70-9c56-9b1fc98244b9 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:57.613676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e5cb6-4bb5-4e5a-ae27-8bc53f6a312c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:57.619871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-d45e5cb6-4bb5-4e5a-ae27-8bc53f6a312c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:57.619871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:57.619871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:57.619871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:57.619871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:57.619871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:57.619871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:57.619871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:57.619871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:57.619871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:57.619871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:57.619871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:57.619871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:57.619871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:57.619871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:57.619871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:57.620769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:57.620769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:57.620769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:57.620769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:57.620769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:57.620769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:57.620769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:57.620769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:57.620769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:57.620769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:57.620769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:57.620769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:57.620769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:57.620769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:57.620769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:57.620769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:57.621562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:57.621562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:57.621562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:57.621562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:57.621562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:57.621562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:57.621562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:57.621562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:57.621562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:57.621562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:57.621562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:39:57.621562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:39:57.622178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 158/309] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:39:57.631526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e5cb6-4bb5-4e5a-ae27-8bc53f6a312c None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:57.653226 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca7ff9-bda0-47e9-8631-d27f15f63089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:57.654104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca7ff9-bda0-47e9-8631-d27f15f63089 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:57.654560 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca7ff9-bda0-47e9-8631-d27f15f63089 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:57.655409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca7ff9-bda0-47e9-8631-d27f15f63089 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:57.669219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca7ff9-bda0-47e9-8631-d27f15f63089 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:57.676332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-f3ca7ff9-bda0-47e9-8631-d27f15f63089 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:39:57.676332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:57.676332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:57.676332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:57.676332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:57.676332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:57.676332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:57.676332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:57.676332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:57.676332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:57.676332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:57.676332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:57.676332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:57.676332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:57.676332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:57.676332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:57.677410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:57.677410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:57.677410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:57.677410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:57.677410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:57.677410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:57.677410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:57.677410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:57.677410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:57.677410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:57.677410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:57.677410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:57.677410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:57.677410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:57.677410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:57.677410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:57.678394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:57.678394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:57.678394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:57.678394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:57.678394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:57.678394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:57.678394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:57.678394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:57.678394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:57.678394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:57.678394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:39:57.678394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:57.681338 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 152/310] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:39:57.687274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca7ff9-bda0-47e9-8631-d27f15f63089 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:57.716107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:57.728542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:57.761436 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e5cb6-4bb5-4e5a-ae27-8bc53f6a312c None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:57.803670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca7ff9-bda0-47e9-8631-d27f15f63089 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:57.811195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dcc8f-7044-48b0-a46f-767c78fe0212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:57.812553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dcc8f-7044-48b0-a46f-767c78fe0212 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:57.812993 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dcc8f-7044-48b0-a46f-767c78fe0212 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:57.813471 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75dcc8f-7044-48b0-a46f-767c78fe0212 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:57.814350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dcc8f-7044-48b0-a46f-767c78fe0212 None admin] RBAC: Adding query filter params (name=member) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:57.815305 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dcc8f-7044-48b0-a46f-767c78fe0212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:57.817644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75dcc8f-7044-48b0-a46f-767c78fe0212 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:57.833949 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 159/311] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:57.834887 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dcc8f-7044-48b0-a46f-767c78fe0212 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:57.847169 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae290e-b9f4-4176-91ef-467f38577841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:57.847782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae290e-b9f4-4176-91ef-467f38577841 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:57.848478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae290e-b9f4-4176-91ef-467f38577841 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:57.848478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae290e-b9f4-4176-91ef-467f38577841 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:57.849130 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae290e-b9f4-4176-91ef-467f38577841 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:57.849429 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae290e-b9f4-4176-91ef-467f38577841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:57.851117 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae290e-b9f4-4176-91ef-467f38577841 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:57.865403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 153/312] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:39:57.870970 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae290e-b9f4-4176-91ef-467f38577841 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:57.978850 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75dcc8f-7044-48b0-a46f-767c78fe0212 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:57.979840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:57.989218 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae290e-b9f4-4176-91ef-467f38577841 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:58.010338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:58.016913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0738a09-7d34-4d4d-82de-bb5061479fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:58.017966 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0738a09-7d34-4d4d-82de-bb5061479fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:58.018650 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0738a09-7d34-4d4d-82de-bb5061479fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:58.019122 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0738a09-7d34-4d4d-82de-bb5061479fa9 None admin] PATH_INFO: `/v3/users/c99130233d9c45daace4c15f47bba1ba` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:58.020612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0738a09-7d34-4d4d-82de-bb5061479fa9 None admin] RBAC: Authorizing `identity:get_user(user_id=c99130233d9c45daace4c15f47bba1ba)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:58.022132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0738a09-7d34-4d4d-82de-bb5061479fa9 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:58.028288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 160/313] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:57 2022] GET /identity/v3/users/c99130233d9c45daace4c15f47bba1ba => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:58.029241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0738a09-7d34-4d4d-82de-bb5061479fa9 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:58.041369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb7c2c-9f63-4f84-b5c5-11188bb191a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:58.042009 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb7c2c-9f63-4f84-b5c5-11188bb191a6 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:58.042285 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb7c2c-9f63-4f84-b5c5-11188bb191a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:58.042591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb7c2c-9f63-4f84-b5c5-11188bb191a6 None admin] PATH_INFO: `/v3/users/21bfe799132e4d30b5afd7d822d61afa` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:58.043783 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb7c2c-9f63-4f84-b5c5-11188bb191a6 None admin] RBAC: Authorizing `identity:get_user(user_id=21bfe799132e4d30b5afd7d822d61afa)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:58.045385 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb7c2c-9f63-4f84-b5c5-11188bb191a6 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:58.050596 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 154/314] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:57 2022] GET /identity/v3/users/21bfe799132e4d30b5afd7d822d61afa => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:58.052916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb7c2c-9f63-4f84-b5c5-11188bb191a6 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:58.151876 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0738a09-7d34-4d4d-82de-bb5061479fa9 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:58.183787 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb7c2c-9f63-4f84-b5c5-11188bb191a6 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:58.189901 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0e877-20c1-4abe-8c98-53384101100f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:58.190627 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0e877-20c1-4abe-8c98-53384101100f None admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:58.190910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0e877-20c1-4abe-8c98-53384101100f None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:58.191188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0e877-20c1-4abe-8c98-53384101100f None admin] PATH_INFO: `/v3/system/users/c99130233d9c45daace4c15f47bba1ba/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:58.192749 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0e877-20c1-4abe-8c98-53384101100f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c99130233d9c45daace4c15f47bba1ba, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:58.194138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0e877-20c1-4abe-8c98-53384101100f None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:58.214532 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 161/315] 10.222.0.65 () {58 vars in 1334 bytes} [Wed Aug 3 12:39:58 2022] PUT /identity/v3/system/users/c99130233d9c45daace4c15f47bba1ba/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:39:58.232840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca3f8d-5e9a-44f9-92d0-e1f6882771a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:58.233479 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca3f8d-5e9a-44f9-92d0-e1f6882771a1 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:58.233759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca3f8d-5e9a-44f9-92d0-e1f6882771a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:58.234031 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca3f8d-5e9a-44f9-92d0-e1f6882771a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:58.271346 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:58.314649 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca3f8d-5e9a-44f9-92d0-e1f6882771a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:58.316147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca3f8d-5e9a-44f9-92d0-e1f6882771a1 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:58.421159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 155/316] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:39:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:39:58.427301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0e877-20c1-4abe-8c98-53384101100f None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:58.493617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:58.534571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0e877-20c1-4abe-8c98-53384101100f None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:58.570871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bceff-4d41-4268-bf9d-f6fe525b059b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:58.571565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bceff-4d41-4268-bf9d-f6fe525b059b None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:58.571798 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bceff-4d41-4268-bf9d-f6fe525b059b None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:58.572072 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bceff-4d41-4268-bf9d-f6fe525b059b None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:58.573250 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bceff-4d41-4268-bf9d-f6fe525b059b None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:58.574718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bceff-4d41-4268-bf9d-f6fe525b059b None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:58.576701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 162/317] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:39:58 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:58.583502 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca3f8d-5e9a-44f9-92d0-e1f6882771a1 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:58.648475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:58.690763 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca3f8d-5e9a-44f9-92d0-e1f6882771a1 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:58.727006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca350b-790f-4c51-a322-a036c1bcd0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:58.727631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca350b-790f-4c51-a322-a036c1bcd0cd None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:58.727911 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca350b-790f-4c51-a322-a036c1bcd0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:58.728185 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca350b-790f-4c51-a322-a036c1bcd0cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:58.728978 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca350b-790f-4c51-a322-a036c1bcd0cd None admin] RBAC: Adding query filter params (role.id=ebfd821c64dd448cae68d5408ae32993, scope.project.id=efec6052310d4c7289f0ce48ac293400, user.id=21bfe799132e4d30b5afd7d822d61afa) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:58.729348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca350b-790f-4c51-a322-a036c1bcd0cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:58.730913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ca350b-790f-4c51-a322-a036c1bcd0cd None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:58.737455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 156/318] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:39:58 2022] GET /identity/v3/role_assignments?user.id=21bfe799132e4d30b5afd7d822d61afa&scope.project.id=efec6052310d4c7289f0ce48ac293400&role.id=ebfd821c64dd448cae68d5408ae32993 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:59.139697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bae119-f4e4-4931-8f38-0d02a507e132 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:59.140064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bae119-f4e4-4931-8f38-0d02a507e132 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:59.140258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bae119-f4e4-4931-8f38-0d02a507e132 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:59.141683 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 163/319] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:59.151137 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b29baa-277d-45fd-9b12-1ce2d68135f6 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:59.151419 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b29baa-277d-45fd-9b12-1ce2d68135f6 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:59.151687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b29baa-277d-45fd-9b12-1ce2d68135f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:59.175476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-60b29baa-277d-45fd-9b12-1ce2d68135f6 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:59.176739 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-60b29baa-277d-45fd-9b12-1ce2d68135f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:59.250578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 157/320] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:39:59.270767 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bae119-f4e4-4931-8f38-0d02a507e132 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:59.336406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d8a52-7cb2-4055-b73a-bc6fbbe12e79 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:59.336699 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d8a52-7cb2-4055-b73a-bc6fbbe12e79 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:59.336953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3d8a52-7cb2-4055-b73a-bc6fbbe12e79 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:59.338321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 158/321] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:39:59.343275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a147d05-16ee-4737-babd-b26fa3ccb80e None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:59.343560 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a147d05-16ee-4737-babd-b26fa3ccb80e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:59.343837 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a147d05-16ee-4737-babd-b26fa3ccb80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:59.363272 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-9a147d05-16ee-4737-babd-b26fa3ccb80e None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:39:59.364502 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-9a147d05-16ee-4737-babd-b26fa3ccb80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:39:59.383014 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bae119-f4e4-4931-8f38-0d02a507e132 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:59.413033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:59.419708 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4298cc5-5316-44d7-a714-6eac4be9c076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:59.420348 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4298cc5-5316-44d7-a714-6eac4be9c076 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:59.420624 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4298cc5-5316-44d7-a714-6eac4be9c076 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:59.420896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4298cc5-5316-44d7-a714-6eac4be9c076 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:59.421515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:59.422028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4298cc5-5316-44d7-a714-6eac4be9c076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:59.424244 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4298cc5-5316-44d7-a714-6eac4be9c076 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:59.426209 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 164/322] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:39:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:59.431602 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4298cc5-5316-44d7-a714-6eac4be9c076 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:59.438536 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 159/323] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:39:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:39:59.459510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a147d05-16ee-4737-babd-b26fa3ccb80e None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:59.534367 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4298cc5-5316-44d7-a714-6eac4be9c076 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:59.569727 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a147d05-16ee-4737-babd-b26fa3ccb80e None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:59.571159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183be9dd-afb5-4b87-ae75-06c0ba2cba21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:59.571846 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183be9dd-afb5-4b87-ae75-06c0ba2cba21 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:59.572134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183be9dd-afb5-4b87-ae75-06c0ba2cba21 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:59.572410 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183be9dd-afb5-4b87-ae75-06c0ba2cba21 None admin] PATH_INFO: `/v3/groups` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:59.573066 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183be9dd-afb5-4b87-ae75-06c0ba2cba21 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:59.574513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183be9dd-afb5-4b87-ae75-06c0ba2cba21 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:59.592637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 165/324] 10.222.0.65 () {60 vars in 1134 bytes} [Wed Aug 3 12:39:59 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:39:59.619194 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046229e7-ff7c-4848-83ae-2ce21772725e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:59.620223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046229e7-ff7c-4848-83ae-2ce21772725e None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:59.620687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046229e7-ff7c-4848-83ae-2ce21772725e None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:59.621131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046229e7-ff7c-4848-83ae-2ce21772725e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:59.633533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046229e7-ff7c-4848-83ae-2ce21772725e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:39:59.639236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-046229e7-ff7c-4848-83ae-2ce21772725e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:39:59.639236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:39:59.639236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:39:59.639236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:39:59.639236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:39:59.639236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:39:59.639236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:39:59.639236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:39:59.639236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:39:59.639236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:39:59.639236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:39:59.639236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:39:59.639236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:39:59.639236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:39:59.639236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:39:59.639236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:39:59.640193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:39:59.640193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:39:59.640193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:39:59.640193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:39:59.640193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:39:59.640193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:39:59.640193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:39:59.640193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:39:59.640193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:39:59.640193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:39:59.640193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:39:59.640193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:39:59.640193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:39:59.640193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:39:59.640193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:39:59.640193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:39:59.640985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:39:59.640985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:39:59.640985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:39:59.640985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:39:59.640985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:39:59.640985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:39:59.640985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:39:59.640985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:39:59.640985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:39:59.640985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:39:59.640985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:39:59.640985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:39:59.643953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 160/325] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:39:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:39:59.649947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183be9dd-afb5-4b87-ae75-06c0ba2cba21 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:59.715732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:59.761998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183be9dd-afb5-4b87-ae75-06c0ba2cba21 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:59.797217 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37384f-ad30-47df-9a72-d7f2bad3f8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:59.797875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37384f-ad30-47df-9a72-d7f2bad3f8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:59.798157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37384f-ad30-47df-9a72-d7f2bad3f8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:59.798429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c37384f-ad30-47df-9a72-d7f2bad3f8d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:59.799191 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37384f-ad30-47df-9a72-d7f2bad3f8d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:39:59.799500 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37384f-ad30-47df-9a72-d7f2bad3f8d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:59.800866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c37384f-ad30-47df-9a72-d7f2bad3f8d4 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:59.809376 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 166/326] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:39:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:59.815192 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046229e7-ff7c-4848-83ae-2ce21772725e None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:39:59.875120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:39:59.930035 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046229e7-ff7c-4848-83ae-2ce21772725e None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:39:59.966826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406b0ef-7c2e-4d1c-bc07-a0f3ff879b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:39:59.967393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406b0ef-7c2e-4d1c-bc07-a0f3ff879b66 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:39:59.967670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406b0ef-7c2e-4d1c-bc07-a0f3ff879b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:39:59.967938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406b0ef-7c2e-4d1c-bc07-a0f3ff879b66 None admin] PATH_INFO: `/v3/users/27215ee848b64056b9ddb890ad6062bf` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:39:59.969124 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406b0ef-7c2e-4d1c-bc07-a0f3ff879b66 None admin] RBAC: Authorizing `identity:get_user(user_id=27215ee848b64056b9ddb890ad6062bf)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:39:59.970637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406b0ef-7c2e-4d1c-bc07-a0f3ff879b66 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:39:59.972544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 161/327] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:39:59 2022] GET /identity/v3/users/27215ee848b64056b9ddb890ad6062bf => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:39:59.978268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37384f-ad30-47df-9a72-d7f2bad3f8d4 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:00.095173 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c37384f-ad30-47df-9a72-d7f2bad3f8d4 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:00.118887 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:00.130994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9bd74-9591-4d35-a8f3-edc1dc673591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:00.132011 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9bd74-9591-4d35-a8f3-edc1dc673591 None admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:00.132694 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9bd74-9591-4d35-a8f3-edc1dc673591 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:00.132969 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9bd74-9591-4d35-a8f3-edc1dc673591 None admin] PATH_INFO: `/v3/system/users/27215ee848b64056b9ddb890ad6062bf/roles/ebfd821c64dd448cae68d5408ae32993` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:00.134442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9bd74-9591-4d35-a8f3-edc1dc673591 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=27215ee848b64056b9ddb890ad6062bf, role_id=ebfd821c64dd448cae68d5408ae32993)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:00.135864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea9bd74-9591-4d35-a8f3-edc1dc673591 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:00.147490 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 167/328] 10.222.0.65 () {58 vars in 1334 bytes} [Wed Aug 3 12:39:59 2022] PUT /identity/v3/system/users/27215ee848b64056b9ddb890ad6062bf/roles/ebfd821c64dd448cae68d5408ae32993 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:40:00.295548 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d17dc3-e625-4e1b-987f-4193517609b4 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:00.295830 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d17dc3-e625-4e1b-987f-4193517609b4 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:00.296221 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d17dc3-e625-4e1b-987f-4193517609b4 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:00.298691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 162/329] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:00.305294 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e89c08-809c-43d8-a0d5-c24fdeb8d17d None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:00.305701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e89c08-809c-43d8-a0d5-c24fdeb8d17d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:00.306076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e89c08-809c-43d8-a0d5-c24fdeb8d17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:00.326349 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-92e89c08-809c-43d8-a0d5-c24fdeb8d17d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:00.327730 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-92e89c08-809c-43d8-a0d5-c24fdeb8d17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:00.405022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 168/330] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:00.429526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d17dc3-e625-4e1b-987f-4193517609b4 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:00.550143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d17dc3-e625-4e1b-987f-4193517609b4 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:00.571459 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:00.591075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9839615-8387-42dc-acdc-935d69643102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:00.591720 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9839615-8387-42dc-acdc-935d69643102 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:00.591996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9839615-8387-42dc-acdc-935d69643102 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:00.592269 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9839615-8387-42dc-acdc-935d69643102 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:00.603854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9839615-8387-42dc-acdc-935d69643102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:00.609568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-b9839615-8387-42dc-acdc-935d69643102 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:40:00.609568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:00.609568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:00.609568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:00.609568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:00.609568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:00.609568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:00.609568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:00.609568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:00.609568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:00.609568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:00.609568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:00.609568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:00.609568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:00.609568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:00.609568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:00.610991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:00.610991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:00.610991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:00.610991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:00.610991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:00.610991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:00.610991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:00.610991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:00.610991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:00.610991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:00.610991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:00.610991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:00.610991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:00.610991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:00.610991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:00.610991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:00.612205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:00.612205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:00.612205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:00.612205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:00.612205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:00.612205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:00.612205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:00.612205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:00.612205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:00.612205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:00.612205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:40:00.612205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:00.612205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 163/331] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:00.617483 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e89c08-809c-43d8-a0d5-c24fdeb8d17d None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:00.728067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e89c08-809c-43d8-a0d5-c24fdeb8d17d None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:00.746222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:00.764275 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47071d1f-a436-4210-8a84-fe12dfd7c4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:00.764902 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47071d1f-a436-4210-8a84-fe12dfd7c4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:00.765183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47071d1f-a436-4210-8a84-fe12dfd7c4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:00.765453 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47071d1f-a436-4210-8a84-fe12dfd7c4e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:00.766273 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47071d1f-a436-4210-8a84-fe12dfd7c4e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:00.766632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47071d1f-a436-4210-8a84-fe12dfd7c4e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:00.768491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47071d1f-a436-4210-8a84-fe12dfd7c4e3 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:00.777585 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 169/332] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:40:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:00.783758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9839615-8387-42dc-acdc-935d69643102 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:00.923244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9839615-8387-42dc-acdc-935d69643102 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:00.972718 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4a16d-5bc6-4621-8276-caa087ea096b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:00.973410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4a16d-5bc6-4621-8276-caa087ea096b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:00.973692 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4a16d-5bc6-4621-8276-caa087ea096b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:00.973959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4a16d-5bc6-4621-8276-caa087ea096b None admin] PATH_INFO: `/v3/users/c99130233d9c45daace4c15f47bba1ba` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:00.975151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4a16d-5bc6-4621-8276-caa087ea096b None admin] RBAC: Authorizing `identity:get_user(user_id=c99130233d9c45daace4c15f47bba1ba)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:00.976582 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4a16d-5bc6-4621-8276-caa087ea096b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:00.978166 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 164/333] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:40:00 2022] GET /identity/v3/users/c99130233d9c45daace4c15f47bba1ba => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:00.983996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47071d1f-a436-4210-8a84-fe12dfd7c4e3 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:01.104442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47071d1f-a436-4210-8a84-fe12dfd7c4e3 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:01.145508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0813180-6722-4a47-8366-36c6bd6cd9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:01.146063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0813180-6722-4a47-8366-36c6bd6cd9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:01.146343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0813180-6722-4a47-8366-36c6bd6cd9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:01.146650 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0813180-6722-4a47-8366-36c6bd6cd9e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:01.147445 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0813180-6722-4a47-8366-36c6bd6cd9e1 None admin] RBAC: Adding query filter params (role.id=0f74302099d9402ba2a922f36cff56ef, user.id=c99130233d9c45daace4c15f47bba1ba, scope.system=all) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:01.147754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0813180-6722-4a47-8366-36c6bd6cd9e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:01.149175 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0813180-6722-4a47-8366-36c6bd6cd9e1 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:01.161995 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 170/334] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Aug 3 12:40:00 2022] GET /identity/v3/role_assignments?user.id=c99130233d9c45daace4c15f47bba1ba&scope.system=all&role.id=0f74302099d9402ba2a922f36cff56ef => generated 510 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:02.126103 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb7d6f-2163-4c6c-8606-d65204e5fa9a None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:02.126642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb7d6f-2163-4c6c-8606-d65204e5fa9a None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:02.126717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb7d6f-2163-4c6c-8606-d65204e5fa9a None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:02.128379 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 165/335] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:02.133707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f1ec4-3698-4598-8cf4-fee45ed7b03f None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:02.134117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f1ec4-3698-4598-8cf4-fee45ed7b03f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:02.134307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f1ec4-3698-4598-8cf4-fee45ed7b03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:02.163516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-0a3f1ec4-3698-4598-8cf4-fee45ed7b03f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:02.164728 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0a3f1ec4-3698-4598-8cf4-fee45ed7b03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:02.202037 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:02.253384 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 171/336] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:02.301770 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb7d6f-2163-4c6c-8606-d65204e5fa9a None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:02.369743 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:02.433357 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb7d6f-2163-4c6c-8606-d65204e5fa9a None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:02.476498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70559e-2f57-4bf6-9a7e-8e37ecb66ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:02.477150 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70559e-2f57-4bf6-9a7e-8e37ecb66ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:02.477431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70559e-2f57-4bf6-9a7e-8e37ecb66ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:02.477760 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce70559e-2f57-4bf6-9a7e-8e37ecb66ed6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:02.493290 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce70559e-2f57-4bf6-9a7e-8e37ecb66ed6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:02.502882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-ce70559e-2f57-4bf6-9a7e-8e37ecb66ed6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:40:02.502882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:02.502882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:02.502882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:02.502882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:02.502882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:02.502882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:02.502882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:02.502882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:02.502882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:02.502882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:02.502882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:02.502882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:02.502882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:02.502882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:02.502882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:02.503836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:02.503836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:02.503836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:02.503836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:02.503836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:02.503836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:02.503836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:02.503836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:02.503836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:02.503836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:02.503836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:02.503836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:02.503836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:02.503836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:02.503836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:02.503836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:02.504630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:02.504630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:02.504630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:02.504630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:02.504630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:02.504630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:02.504630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:02.504630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:02.504630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:02.504630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:02.504630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:40:02.504630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:02.504630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 166/337] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:02.505301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d787b-8719-41fc-8984-480e63edc5d1 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:02.505301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d787b-8719-41fc-8984-480e63edc5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:02.505301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d787b-8719-41fc-8984-480e63edc5d1 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:02.506428 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 172/338] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:40:02.512135 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70559e-2f57-4bf6-9a7e-8e37ecb66ed6 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:02.515629 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f8d2e-0544-4106-ad40-b890a0b42533 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:02.516216 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f8d2e-0544-4106-ad40-b890a0b42533 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:02.517184 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f8d2e-0544-4106-ad40-b890a0b42533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:02.536783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-045f8d2e-0544-4106-ad40-b890a0b42533 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:02.538344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-045f8d2e-0544-4106-ad40-b890a0b42533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:02.653566 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 173/339] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:40:02.690289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce70559e-2f57-4bf6-9a7e-8e37ecb66ed6 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:02.692595 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:02.699889 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f8d2e-0544-4106-ad40-b890a0b42533 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:02.751178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda1c46-ad69-4eca-add4-493b6c3d2c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:02.751816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda1c46-ad69-4eca-add4-493b6c3d2c0f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:02.752094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda1c46-ad69-4eca-add4-493b6c3d2c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:02.753670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda1c46-ad69-4eca-add4-493b6c3d2c0f None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:02.753670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda1c46-ad69-4eca-add4-493b6c3d2c0f None admin] RBAC: Adding query filter params (name=member) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:02.753670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda1c46-ad69-4eca-add4-493b6c3d2c0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:02.754765 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda1c46-ad69-4eca-add4-493b6c3d2c0f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:02.764189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 167/340] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:40:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:02.769794 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda1c46-ad69-4eca-add4-493b6c3d2c0f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:02.825745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:02.832951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f8d2e-0544-4106-ad40-b890a0b42533 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:02.882309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea3230-4c83-4dc6-a396-8b614ecbd4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:02.883240 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea3230-4c83-4dc6-a396-8b614ecbd4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:02.883724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea3230-4c83-4dc6-a396-8b614ecbd4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:02.884238 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea3230-4c83-4dc6-a396-8b614ecbd4f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:02.895380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda1c46-ad69-4eca-add4-493b6c3d2c0f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:02.898477 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea3230-4c83-4dc6-a396-8b614ecbd4f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:02.903956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-47ea3230-4c83-4dc6-a396-8b614ecbd4f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:40:02.903956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:02.903956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:02.903956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:02.903956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:02.903956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:02.903956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:02.903956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:02.903956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:02.903956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:02.903956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:02.903956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:02.903956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:02.903956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:02.903956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:02.903956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:02.904855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:02.904855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:02.904855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:02.904855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:02.904855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:02.904855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:02.904855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:02.904855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:02.904855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:02.904855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:02.904855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:02.904855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:02.904855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:02.904855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:02.904855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:02.904855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:02.905717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:02.905717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:02.905717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:02.905717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:02.905717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:02.905717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:02.905717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:02.905717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:02.905717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:02.905717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:02.905717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:40:02.905717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:02.909083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 174/341] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:40:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:02.911798 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f604140-2685-40df-ab93-0ad25aa48f00 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:02.912346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f604140-2685-40df-ab93-0ad25aa48f00 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:02.912825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f604140-2685-40df-ab93-0ad25aa48f00 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:02.918508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 175/342] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:02.920950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2536999-e197-4c64-8fea-32f5a61e9aa8 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:02.921446 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2536999-e197-4c64-8fea-32f5a61e9aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:02.921955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2536999-e197-4c64-8fea-32f5a61e9aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:02.939125 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0abbcc-2888-4dc9-a1c8-d4b946e59aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:02.939802 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0abbcc-2888-4dc9-a1c8-d4b946e59aeb None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:02.940081 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0abbcc-2888-4dc9-a1c8-d4b946e59aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:02.940403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0abbcc-2888-4dc9-a1c8-d4b946e59aeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:02.943459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-d2536999-e197-4c64-8fea-32f5a61e9aa8 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:02.944854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d2536999-e197-4c64-8fea-32f5a61e9aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:02.978262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:03.018146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0abbcc-2888-4dc9-a1c8-d4b946e59aeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:03.018333 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 176/343] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:03.019445 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2536999-e197-4c64-8fea-32f5a61e9aa8 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:03.023418 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0abbcc-2888-4dc9-a1c8-d4b946e59aeb None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:03.113539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:03.131216 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2536999-e197-4c64-8fea-32f5a61e9aa8 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:03.160659 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 168/344] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 12:40:03.162313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0abbcc-2888-4dc9-a1c8-d4b946e59aeb None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:03.167220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695032bd-c8d9-49a5-9307-4a945118999b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:03.167973 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695032bd-c8d9-49a5-9307-4a945118999b None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:03.168377 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695032bd-c8d9-49a5-9307-4a945118999b None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:03.168754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695032bd-c8d9-49a5-9307-4a945118999b None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:03.169593 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695032bd-c8d9-49a5-9307-4a945118999b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:03.169999 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695032bd-c8d9-49a5-9307-4a945118999b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:03.171526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695032bd-c8d9-49a5-9307-4a945118999b None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:03.181200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 177/345] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:03.182127 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695032bd-c8d9-49a5-9307-4a945118999b None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:03.237652 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:03.278946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0abbcc-2888-4dc9-a1c8-d4b946e59aeb None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:03.299163 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695032bd-c8d9-49a5-9307-4a945118999b None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:03.317026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3003bbd-cd93-4cb9-aa05-5c576d4c067e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:03.317933 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3003bbd-cd93-4cb9-aa05-5c576d4c067e None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:03.318354 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3003bbd-cd93-4cb9-aa05-5c576d4c067e None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:03.318808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3003bbd-cd93-4cb9-aa05-5c576d4c067e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:03.330950 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3003bbd-cd93-4cb9-aa05-5c576d4c067e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:03.336558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-d3003bbd-cd93-4cb9-aa05-5c576d4c067e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:40:03.336558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:03.336558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:03.336558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:03.336558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:03.336558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:03.336558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:03.336558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:03.336558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:03.336558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:03.336558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:03.336558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:03.336558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:03.336558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:03.336558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:03.336558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:03.337470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:03.337470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:03.337470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:03.337470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:03.337470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:03.337470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:03.337470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:03.337470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:03.337470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:03.337470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:03.337470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:03.337470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:03.337470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:03.337470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:03.337470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:03.337470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:03.338367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:03.338367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:03.338367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:03.338367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:03.338367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:03.338367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:03.338367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:03.338367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:03.338367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:03.338367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:03.338367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 03 12:40:03.338367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:03.339696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31425983-64c1-43cf-b9e4-afd37d216f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:03.340414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31425983-64c1-43cf-b9e4-afd37d216f32 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:03.340780 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31425983-64c1-43cf-b9e4-afd37d216f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:03.341130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31425983-64c1-43cf-b9e4-afd37d216f32 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:03.341541 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 169/346] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:03.342384 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31425983-64c1-43cf-b9e4-afd37d216f32 None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:03.342843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3003bbd-cd93-4cb9-aa05-5c576d4c067e None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:03.343998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31425983-64c1-43cf-b9e4-afd37d216f32 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:03.346574 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 178/347] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:40:03 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:03.351922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31425983-64c1-43cf-b9e4-afd37d216f32 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:03.373228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:03.449560 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3003bbd-cd93-4cb9-aa05-5c576d4c067e None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:03.469859 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31425983-64c1-43cf-b9e4-afd37d216f32 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:03.477426 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:03.488112 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28023aca-0f57-4162-bc1b-cf63032def1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:03.489029 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28023aca-0f57-4162-bc1b-cf63032def1c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:03.489514 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28023aca-0f57-4162-bc1b-cf63032def1c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:03.489987 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28023aca-0f57-4162-bc1b-cf63032def1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:03.510138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0fc22-a49b-44c9-9604-a0d0770e4fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:03.511592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0fc22-a49b-44c9-9604-a0d0770e4fdc None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:03.511592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0fc22-a49b-44c9-9604-a0d0770e4fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:03.511592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0fc22-a49b-44c9-9604-a0d0770e4fdc None admin] PATH_INFO: `/v3/groups/a213174fbe0548ffa7eca114dee63eb7` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:03.518860 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0fc22-a49b-44c9-9604-a0d0770e4fdc None admin] RBAC: Authorizing `identity:get_group(group_id=a213174fbe0548ffa7eca114dee63eb7)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:03.520422 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb0fc22-a49b-44c9-9604-a0d0770e4fdc None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:03.522609 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 179/348] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 3 12:40:03 2022] GET /identity/v3/groups/a213174fbe0548ffa7eca114dee63eb7 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:03.523484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0fc22-a49b-44c9-9604-a0d0770e4fdc None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:03.565075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28023aca-0f57-4162-bc1b-cf63032def1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:03.566928 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28023aca-0f57-4162-bc1b-cf63032def1c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:03.627510 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:03.650527 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0fc22-a49b-44c9-9604-a0d0770e4fdc None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:03.674563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 170/349] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:40:03.675754 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28023aca-0f57-4162-bc1b-cf63032def1c None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:03.694193 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef6580-31bf-4554-bba0-aef7dfa87d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:03.694863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef6580-31bf-4554-bba0-aef7dfa87d70 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:03.695146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef6580-31bf-4554-bba0-aef7dfa87d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:03.695624 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef6580-31bf-4554-bba0-aef7dfa87d70 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:03.696485 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef6580-31bf-4554-bba0-aef7dfa87d70 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:03.696798 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef6580-31bf-4554-bba0-aef7dfa87d70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:03.698271 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef6580-31bf-4554-bba0-aef7dfa87d70 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:03.707836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:03.709431 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 180/350] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:40:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:03.710425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef6580-31bf-4554-bba0-aef7dfa87d70 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:03.786883 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28023aca-0f57-4162-bc1b-cf63032def1c None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:03.823580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef6580-31bf-4554-bba0-aef7dfa87d70 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:03.824218 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ee37b-a200-4bc2-ac0a-8fce16455b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:03.825086 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ee37b-a200-4bc2-ac0a-8fce16455b56 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:03.825574 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ee37b-a200-4bc2-ac0a-8fce16455b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:03.826059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ee37b-a200-4bc2-ac0a-8fce16455b56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:03.827118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94ee37b-a200-4bc2-ac0a-8fce16455b56 None admin] RBAC: Adding query filter params (group.id=a213174fbe0548ffa7eca114dee63eb7, role.id=0f74302099d9402ba2a922f36cff56ef, scope.project.id=22d0a79f59184a36a955aa3a842c3119) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:03.827629 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94ee37b-a200-4bc2-ac0a-8fce16455b56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:03.829926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94ee37b-a200-4bc2-ac0a-8fce16455b56 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:03.836164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 171/351] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 3 12:40:03 2022] GET /identity/v3/role_assignments?group.id=a213174fbe0548ffa7eca114dee63eb7&scope.project.id=22d0a79f59184a36a955aa3a842c3119&role.id=0f74302099d9402ba2a922f36cff56ef => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:03.837206 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ee37b-a200-4bc2-ac0a-8fce16455b56 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:03.868693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda513e4-f472-4201-a4fa-dd4e2d7156dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:03.869365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda513e4-f472-4201-a4fa-dd4e2d7156dd None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:03.869646 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda513e4-f472-4201-a4fa-dd4e2d7156dd None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:03.869927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda513e4-f472-4201-a4fa-dd4e2d7156dd None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:03.871340 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda513e4-f472-4201-a4fa-dd4e2d7156dd None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:03.878570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda513e4-f472-4201-a4fa-dd4e2d7156dd None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:03.880764 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 181/352] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:40:03 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:03.886335 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda513e4-f472-4201-a4fa-dd4e2d7156dd None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:03.901097 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:03.945393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:03.956298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ee37b-a200-4bc2-ac0a-8fce16455b56 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:03.994472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd063487-f68d-47c1-8b76-50f4beec4604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:03.995557 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd063487-f68d-47c1-8b76-50f4beec4604 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:03.996039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd063487-f68d-47c1-8b76-50f4beec4604 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:03.996511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd063487-f68d-47c1-8b76-50f4beec4604 None admin] PATH_INFO: `/v3/users/27215ee848b64056b9ddb890ad6062bf` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:03.997942 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd063487-f68d-47c1-8b76-50f4beec4604 None admin] RBAC: Authorizing `identity:get_user(user_id=27215ee848b64056b9ddb890ad6062bf)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:03.999758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd063487-f68d-47c1-8b76-50f4beec4604 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:04.002662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 172/353] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:40:03 2022] GET /identity/v3/users/27215ee848b64056b9ddb890ad6062bf => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:04.004129 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a36b7-8b3d-4118-9acf-9d6626eda227 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:04.004615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a36b7-8b3d-4118-9acf-9d6626eda227 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:04.005113 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a36b7-8b3d-4118-9acf-9d6626eda227 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:04.006707 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 173/354] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:04.007608 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda513e4-f472-4201-a4fa-dd4e2d7156dd None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:04.015926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762845ff-8f8a-497f-be39-874fc02551f2 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:04.016888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762845ff-8f8a-497f-be39-874fc02551f2 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:04.017140 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762845ff-8f8a-497f-be39-874fc02551f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:04.046019 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b7330-3e85-462b-8b8e-f80a22ae58d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:04.046646 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b7330-3e85-462b-8b8e-f80a22ae58d8 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:04.046928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b7330-3e85-462b-8b8e-f80a22ae58d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:04.047201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b7330-3e85-462b-8b8e-f80a22ae58d8 None admin] PATH_INFO: `/v3/groups/a213174fbe0548ffa7eca114dee63eb7` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:04.049080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b7330-3e85-462b-8b8e-f80a22ae58d8 None admin] RBAC: Authorizing `identity:get_group(group_id=a213174fbe0548ffa7eca114dee63eb7)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:04.050639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b7330-3e85-462b-8b8e-f80a22ae58d8 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:04.052634 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 182/355] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 3 12:40:03 2022] GET /identity/v3/groups/a213174fbe0548ffa7eca114dee63eb7 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:04.053425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b7330-3e85-462b-8b8e-f80a22ae58d8 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:04.064330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-762845ff-8f8a-497f-be39-874fc02551f2 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:04.065596 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-762845ff-8f8a-497f-be39-874fc02551f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:04.159252 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:04.198037 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b7330-3e85-462b-8b8e-f80a22ae58d8 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:04.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 174/356] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:04.218466 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762845ff-8f8a-497f-be39-874fc02551f2 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:04.264179 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66964b3c-05b4-417c-922f-65c7558836da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:04.265926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66964b3c-05b4-417c-922f-65c7558836da None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:04.266260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66964b3c-05b4-417c-922f-65c7558836da None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:04.266604 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66964b3c-05b4-417c-922f-65c7558836da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:04.267527 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66964b3c-05b4-417c-922f-65c7558836da None admin] RBAC: Adding query filter params (role.id=ebfd821c64dd448cae68d5408ae32993, user.id=27215ee848b64056b9ddb890ad6062bf, scope.system=all) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:04.267841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66964b3c-05b4-417c-922f-65c7558836da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:04.269439 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66964b3c-05b4-417c-922f-65c7558836da None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:04.293920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 183/357] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Aug 3 12:40:04 2022] GET /identity/v3/role_assignments?user.id=27215ee848b64056b9ddb890ad6062bf&scope.system=all&role.id=ebfd821c64dd448cae68d5408ae32993 => generated 510 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:04.294943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66964b3c-05b4-417c-922f-65c7558836da None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:04.407627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:04.427084 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762845ff-8f8a-497f-be39-874fc02551f2 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:04.476809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66964b3c-05b4-417c-922f-65c7558836da None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:04.485451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aacec7-3372-4b6e-bafb-85eb89322477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:04.486085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aacec7-3372-4b6e-bafb-85eb89322477 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:04.486368 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aacec7-3372-4b6e-bafb-85eb89322477 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:04.486668 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aacec7-3372-4b6e-bafb-85eb89322477 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:04.487475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aacec7-3372-4b6e-bafb-85eb89322477 None admin] RBAC: Adding query filter params (group.id=a213174fbe0548ffa7eca114dee63eb7, role.id=79ff845a41e241229433560e2b0d6fe5, scope.project.id=22d0a79f59184a36a955aa3a842c3119) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:04.487785 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aacec7-3372-4b6e-bafb-85eb89322477 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:04.489222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aacec7-3372-4b6e-bafb-85eb89322477 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:04.503272 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 175/358] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 3 12:40:04 2022] GET /identity/v3/role_assignments?group.id=a213174fbe0548ffa7eca114dee63eb7&scope.project.id=22d0a79f59184a36a955aa3a842c3119&role.id=79ff845a41e241229433560e2b0d6fe5 => generated 260 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:04.504366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590d71c-512a-4e00-869c-3f2ccafd8240 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:04.504647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590d71c-512a-4e00-869c-3f2ccafd8240 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:04.504916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590d71c-512a-4e00-869c-3f2ccafd8240 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:04.506133 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 176/359] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:04.511758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75fc74-320b-4b9c-acb5-835c7bfb4f43 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:04.512061 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75fc74-320b-4b9c-acb5-835c7bfb4f43 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:04.512301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75fc74-320b-4b9c-acb5-835c7bfb4f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:04.530429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b8bfc-830c-42b3-b962-891cc33371bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:04.531561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b8bfc-830c-42b3-b962-891cc33371bd None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:04.532584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b8bfc-830c-42b3-b962-891cc33371bd None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:04.533081 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b8bfc-830c-42b3-b962-891cc33371bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:04.545857 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:04.549168 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-4d75fc74-320b-4b9c-acb5-835c7bfb4f43 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:04.550471 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-4d75fc74-320b-4b9c-acb5-835c7bfb4f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:04.565470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b8bfc-830c-42b3-b962-891cc33371bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:04.574879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-657b8bfc-830c-42b3-b962-891cc33371bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:40:04.574879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:04.574879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:04.574879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:04.574879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:04.574879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:04.574879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:04.574879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:04.574879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:04.574879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:04.574879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:04.574879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:04.574879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:04.574879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:04.574879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:04.574879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:04.575845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:04.575845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:04.575845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:04.575845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:04.575845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:04.575845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:04.575845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:04.575845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:04.575845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:04.575845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:04.575845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:04.575845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:04.575845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:04.575845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:04.575845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:04.575845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:04.576654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:04.576654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:04.576654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:04.576654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:04.576654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:04.576654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:04.576654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:04.576654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:04.576654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:04.576654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:04.576654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:40:04.576654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:04.579762 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 184/360] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:04.585264 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b8bfc-830c-42b3-b962-891cc33371bd None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:04.712022 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 177/361] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:40:04.749361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75fc74-320b-4b9c-acb5-835c7bfb4f43 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:04.757858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b8bfc-830c-42b3-b962-891cc33371bd None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:04.797110 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871543e-e259-474b-8a94-651b1be75d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:04.798443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871543e-e259-474b-8a94-651b1be75d43 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:04.798957 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871543e-e259-474b-8a94-651b1be75d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:04.799716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871543e-e259-474b-8a94-651b1be75d43 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:04.800642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b871543e-e259-474b-8a94-651b1be75d43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:04.801144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b871543e-e259-474b-8a94-651b1be75d43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:04.803083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b871543e-e259-474b-8a94-651b1be75d43 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:04.814159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 185/362] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:40:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:04.819555 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871543e-e259-474b-8a94-651b1be75d43 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:04.883405 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:04.897341 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:04.908300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75fc74-320b-4b9c-acb5-835c7bfb4f43 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:04.945908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5ca45-478b-49e9-81ef-564266a1a852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:04.946871 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5ca45-478b-49e9-81ef-564266a1a852 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:04.947343 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5ca45-478b-49e9-81ef-564266a1a852 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:04.947757 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5ca45-478b-49e9-81ef-564266a1a852 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:04.951364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871543e-e259-474b-8a94-651b1be75d43 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:04.959911 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb5ca45-478b-49e9-81ef-564266a1a852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:04.965640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-9cb5ca45-478b-49e9-81ef-564266a1a852 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:40:04.965640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:04.965640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:04.965640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:04.965640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:04.965640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:04.965640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:04.965640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:04.965640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:04.965640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:04.965640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:04.965640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:04.965640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:04.965640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:04.965640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:04.965640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:04.966724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:04.966724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:04.966724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:04.966724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:04.966724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:04.966724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:04.966724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:04.966724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:04.966724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:04.966724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:04.966724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:04.966724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:04.966724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:04.966724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:04.966724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:04.966724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:04.967566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:04.967566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:04.967566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:04.967566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:04.967566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:04.967566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:04.967566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:04.967566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:04.967566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:04.967566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:04.967566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:40:04.967566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:04.970762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 178/363] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:40:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:04.990451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5ca45-478b-49e9-81ef-564266a1a852 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:04.994193 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb05a5a-00d2-4143-a1be-77092cabbecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:04.994863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb05a5a-00d2-4143-a1be-77092cabbecb None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:04.995141 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb05a5a-00d2-4143-a1be-77092cabbecb None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:04.998000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb05a5a-00d2-4143-a1be-77092cabbecb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:05.077170 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb05a5a-00d2-4143-a1be-77092cabbecb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:05.079228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb05a5a-00d2-4143-a1be-77092cabbecb None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:05.142885 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:05.154182 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5ca45-478b-49e9-81ef-564266a1a852 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:05.188605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 186/364] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 12:40:05.193989 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb05a5a-00d2-4143-a1be-77092cabbecb None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:05.240935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532512f-20e6-4901-b741-35aceefc2619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:05.242425 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532512f-20e6-4901-b741-35aceefc2619 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:05.242912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532512f-20e6-4901-b741-35aceefc2619 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:05.243330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532512f-20e6-4901-b741-35aceefc2619 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:05.244206 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532512f-20e6-4901-b741-35aceefc2619 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:05.244742 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532512f-20e6-4901-b741-35aceefc2619 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:05.246749 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532512f-20e6-4901-b741-35aceefc2619 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:05.259369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 179/365] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:05.260881 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd61bf0-2a55-4459-9ca1-b793de01dd36 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:05.261307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd61bf0-2a55-4459-9ca1-b793de01dd36 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:05.261722 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd61bf0-2a55-4459-9ca1-b793de01dd36 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:05.263313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 180/366] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:05.270543 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd61bf0-2a55-4459-9ca1-b793de01dd36 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:05.277679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:05.311245 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb05a5a-00d2-4143-a1be-77092cabbecb None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:05.357035 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d49e0d-4d0a-48bc-b7e8-d46137972bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:05.358007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d49e0d-4d0a-48bc-b7e8-d46137972bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:05.358393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d49e0d-4d0a-48bc-b7e8-d46137972bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:05.358813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d49e0d-4d0a-48bc-b7e8-d46137972bf7 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:05.360917 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d49e0d-4d0a-48bc-b7e8-d46137972bf7 None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:05.362660 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d49e0d-4d0a-48bc-b7e8-d46137972bf7 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:05.365517 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 187/367] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:40:05 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:05.366706 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb48584-592b-4450-ba5a-8e593b7cf90b None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:05.366994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb48584-592b-4450-ba5a-8e593b7cf90b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:05.367270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb48584-592b-4450-ba5a-8e593b7cf90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:05.390388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-bcb48584-592b-4450-ba5a-8e593b7cf90b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:05.392568 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-bcb48584-592b-4450-ba5a-8e593b7cf90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:05.402776 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd61bf0-2a55-4459-9ca1-b793de01dd36 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:05.419048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:05.448825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c986885-f55e-42c3-a711-9ddc2eb54a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:05.450280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c986885-f55e-42c3-a711-9ddc2eb54a16 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:05.450592 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c986885-f55e-42c3-a711-9ddc2eb54a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:05.450873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c986885-f55e-42c3-a711-9ddc2eb54a16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:05.499747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 188/368] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:05.501752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb48584-592b-4450-ba5a-8e593b7cf90b None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:05.544512 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c986885-f55e-42c3-a711-9ddc2eb54a16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:05.547163 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c986885-f55e-42c3-a711-9ddc2eb54a16 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:05.550871 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:05.624243 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb48584-592b-4450-ba5a-8e593b7cf90b None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:05.661717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4349a4-39b7-49ca-a8f7-248e63a81422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:05.662468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4349a4-39b7-49ca-a8f7-248e63a81422 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:05.662773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4349a4-39b7-49ca-a8f7-248e63a81422 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:05.663048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4349a4-39b7-49ca-a8f7-248e63a81422 None admin] PATH_INFO: `/v3/groups/a213174fbe0548ffa7eca114dee63eb7` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:05.664200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4349a4-39b7-49ca-a8f7-248e63a81422 None admin] RBAC: Authorizing `identity:get_group(group_id=a213174fbe0548ffa7eca114dee63eb7)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:05.665871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4349a4-39b7-49ca-a8f7-248e63a81422 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:05.667696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 189/369] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 3 12:40:05 2022] GET /identity/v3/groups/a213174fbe0548ffa7eca114dee63eb7 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:05.668478 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4349a4-39b7-49ca-a8f7-248e63a81422 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:05.672366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 181/370] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:40:05.673629 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c986885-f55e-42c3-a711-9ddc2eb54a16 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:05.700197 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:05.781368 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4349a4-39b7-49ca-a8f7-248e63a81422 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:05.786372 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c986885-f55e-42c3-a711-9ddc2eb54a16 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:05.789021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:05.820107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354115e1-008f-4c59-8201-5fdb78466b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:05.821090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354115e1-008f-4c59-8201-5fdb78466b04 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:05.821549 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354115e1-008f-4c59-8201-5fdb78466b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:05.822103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354115e1-008f-4c59-8201-5fdb78466b04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:05.825872 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8905ae-a204-49d9-8f6d-c89acf2d0f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:05.826700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8905ae-a204-49d9-8f6d-c89acf2d0f18 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:05.826984 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8905ae-a204-49d9-8f6d-c89acf2d0f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:05.827258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8905ae-a204-49d9-8f6d-c89acf2d0f18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:05.828063 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8905ae-a204-49d9-8f6d-c89acf2d0f18 None admin] RBAC: Adding query filter params (group.id=a213174fbe0548ffa7eca114dee63eb7, role.id=0f74302099d9402ba2a922f36cff56ef, scope.project.id=efec6052310d4c7289f0ce48ac293400) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:05.828450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8905ae-a204-49d9-8f6d-c89acf2d0f18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:05.830337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8905ae-a204-49d9-8f6d-c89acf2d0f18 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:05.835441 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354115e1-008f-4c59-8201-5fdb78466b04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:05.836681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 182/371] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 3 12:40:05 2022] GET /identity/v3/role_assignments?group.id=a213174fbe0548ffa7eca114dee63eb7&scope.project.id=efec6052310d4c7289f0ce48ac293400&role.id=0f74302099d9402ba2a922f36cff56ef => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:05.837545 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8905ae-a204-49d9-8f6d-c89acf2d0f18 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:05.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-354115e1-008f-4c59-8201-5fdb78466b04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:05.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:05.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:05.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:05.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:05.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:05.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:05.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:05.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:05.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:05.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:05.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:05.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:05.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:05.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:05.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:05.844960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:05.844960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:05.844960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:05.844960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:05.844960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:05.844960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:05.844960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:05.844960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:05.844960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:05.844960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:05.844960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:05.844960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:05.844960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:05.844960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:05.844960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:05.844960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:05.845770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:05.845770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:05.845770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:05.845770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:05.845770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:05.845770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:05.845770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:05.845770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:05.845770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:05.845770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:05.845770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:05.845770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:05.848390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 190/372] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:40:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:05.855580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354115e1-008f-4c59-8201-5fdb78466b04 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:05.986263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:05.989991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8905ae-a204-49d9-8f6d-c89acf2d0f18 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:06.007132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354115e1-008f-4c59-8201-5fdb78466b04 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:06.037698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c450e1e-b0b9-4458-ad0b-c783947a5194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:06.039218 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c450e1e-b0b9-4458-ad0b-c783947a5194 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:06.039672 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c450e1e-b0b9-4458-ad0b-c783947a5194 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:06.040248 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c450e1e-b0b9-4458-ad0b-c783947a5194 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:06.042038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c450e1e-b0b9-4458-ad0b-c783947a5194 None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:06.044159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c450e1e-b0b9-4458-ad0b-c783947a5194 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:06.046642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 183/373] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:40:05 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:06.049781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3014003-f8d7-4ffa-8838-fcda4dfe247c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:06.050129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3014003-f8d7-4ffa-8838-fcda4dfe247c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:06.050527 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3014003-f8d7-4ffa-8838-fcda4dfe247c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:06.050890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3014003-f8d7-4ffa-8838-fcda4dfe247c None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:06.052393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c450e1e-b0b9-4458-ad0b-c783947a5194 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:06.052893 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3014003-f8d7-4ffa-8838-fcda4dfe247c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:06.053283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3014003-f8d7-4ffa-8838-fcda4dfe247c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:06.054895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3014003-f8d7-4ffa-8838-fcda4dfe247c None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:06.066408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 191/374] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:06.073133 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3014003-f8d7-4ffa-8838-fcda4dfe247c None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:06.106199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:06.212533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c450e1e-b0b9-4458-ad0b-c783947a5194 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:06.237843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3014003-f8d7-4ffa-8838-fcda4dfe247c None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:06.260923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f99be2-12f5-4929-a7d4-b86178f8a12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:06.262577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f99be2-12f5-4929-a7d4-b86178f8a12d None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:06.263071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f99be2-12f5-4929-a7d4-b86178f8a12d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:06.263830 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f99be2-12f5-4929-a7d4-b86178f8a12d None admin] PATH_INFO: `/v3/groups/a213174fbe0548ffa7eca114dee63eb7` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:06.265906 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f99be2-12f5-4929-a7d4-b86178f8a12d None admin] RBAC: Authorizing `identity:get_group(group_id=a213174fbe0548ffa7eca114dee63eb7)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:06.269145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f99be2-12f5-4929-a7d4-b86178f8a12d None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:06.272122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 184/375] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 3 12:40:06 2022] GET /identity/v3/groups/a213174fbe0548ffa7eca114dee63eb7 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:06.277899 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f99be2-12f5-4929-a7d4-b86178f8a12d None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:06.281366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376c8d2-6287-46ab-a93b-ca390b0a40d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:06.282186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376c8d2-6287-46ab-a93b-ca390b0a40d4 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:06.282535 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376c8d2-6287-46ab-a93b-ca390b0a40d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:06.282826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376c8d2-6287-46ab-a93b-ca390b0a40d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:06.306761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:06.402063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376c8d2-6287-46ab-a93b-ca390b0a40d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:06.403564 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376c8d2-6287-46ab-a93b-ca390b0a40d4 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:06.449500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f99be2-12f5-4929-a7d4-b86178f8a12d None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:06.453071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:06.532151 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 192/376] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 12:40:06.536062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f105b15-df42-4dce-8701-bd3705f4e706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:06.536062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f105b15-df42-4dce-8701-bd3705f4e706 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:06.536062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f105b15-df42-4dce-8701-bd3705f4e706 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:06.536062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f105b15-df42-4dce-8701-bd3705f4e706 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:06.536434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f105b15-df42-4dce-8701-bd3705f4e706 None admin] RBAC: Adding query filter params (group.id=a213174fbe0548ffa7eca114dee63eb7, role.id=79ff845a41e241229433560e2b0d6fe5, scope.project.id=efec6052310d4c7289f0ce48ac293400) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:06.536434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f105b15-df42-4dce-8701-bd3705f4e706 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:06.537617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f105b15-df42-4dce-8701-bd3705f4e706 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:06.539151 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376c8d2-6287-46ab-a93b-ca390b0a40d4 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:06.544191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 185/377] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 3 12:40:06 2022] GET /identity/v3/role_assignments?group.id=a213174fbe0548ffa7eca114dee63eb7&scope.project.id=efec6052310d4c7289f0ce48ac293400&role.id=79ff845a41e241229433560e2b0d6fe5 => generated 260 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:06.624169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:06.662754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376c8d2-6287-46ab-a93b-ca390b0a40d4 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:06.717717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a35d37-47c1-4690-b376-6c357fd9a77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:06.718379 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a35d37-47c1-4690-b376-6c357fd9a77a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:06.718688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a35d37-47c1-4690-b376-6c357fd9a77a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:06.718966 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a35d37-47c1-4690-b376-6c357fd9a77a None admin] PATH_INFO: `/v3/projects/87eaffc3c69e48b2bb1ac115a06dc966` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:06.723339 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a35d37-47c1-4690-b376-6c357fd9a77a None admin] RBAC: Authorizing `identity:get_project(project_id=87eaffc3c69e48b2bb1ac115a06dc966)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:06.724905 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a35d37-47c1-4690-b376-6c357fd9a77a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:06.726634 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 193/378] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:40:06 2022] GET /identity/v3/projects/87eaffc3c69e48b2bb1ac115a06dc966 => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:06.736258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f105b15-df42-4dce-8701-bd3705f4e706 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:06.802127 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231a6bf-81c1-4c6b-9cfc-ba4798f44931 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:06.802448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231a6bf-81c1-4c6b-9cfc-ba4798f44931 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:06.802765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231a6bf-81c1-4c6b-9cfc-ba4798f44931 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:06.804360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 194/379] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:06.811019 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d06c3e-3542-41c9-9a02-c02c2d4c6290 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:06.811308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d06c3e-3542-41c9-9a02-c02c2d4c6290 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:06.811589 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d06c3e-3542-41c9-9a02-c02c2d4c6290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:06.833239 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-82d06c3e-3542-41c9-9a02-c02c2d4c6290 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:06.834768 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-82d06c3e-3542-41c9-9a02-c02c2d4c6290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:06.883139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f105b15-df42-4dce-8701-bd3705f4e706 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:06.945000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5e73b-96cb-4a8b-81d5-919c6658beff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:06.945717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5e73b-96cb-4a8b-81d5-919c6658beff None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:06.946005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5e73b-96cb-4a8b-81d5-919c6658beff None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:06.946280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5e73b-96cb-4a8b-81d5-919c6658beff None admin] PATH_INFO: `/v3/groups/1e40589035964a398acba1dd30ca3338` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:06.956908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5e73b-96cb-4a8b-81d5-919c6658beff None admin] RBAC: Authorizing `identity:get_group(group_id=1e40589035964a398acba1dd30ca3338)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:06.958747 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d5e73b-96cb-4a8b-81d5-919c6658beff None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:06.961238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 186/380] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 3 12:40:06 2022] GET /identity/v3/groups/1e40589035964a398acba1dd30ca3338 => generated 229 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:06.967315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5e73b-96cb-4a8b-81d5-919c6658beff None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:06.972775 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 195/381] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:40:07.001219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:07.011877 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d06c3e-3542-41c9-9a02-c02c2d4c6290 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:07.080265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:07.103048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5e73b-96cb-4a8b-81d5-919c6658beff None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:07.164170 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d06c3e-3542-41c9-9a02-c02c2d4c6290 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:07.164581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0f589-119f-421a-afa2-981765dadff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:07.165215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0f589-119f-421a-afa2-981765dadff6 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:07.165619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0f589-119f-421a-afa2-981765dadff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:07.165896 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0f589-119f-421a-afa2-981765dadff6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:07.166724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0f589-119f-421a-afa2-981765dadff6 None admin] RBAC: Adding query filter params (group.id=1e40589035964a398acba1dd30ca3338, role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=87eaffc3c69e48b2bb1ac115a06dc966) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:07.167036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0f589-119f-421a-afa2-981765dadff6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:07.168660 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e0f589-119f-421a-afa2-981765dadff6 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:07.175333 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 187/382] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 3 12:40:06 2022] GET /identity/v3/role_assignments?group.id=1e40589035964a398acba1dd30ca3338&scope.project.id=87eaffc3c69e48b2bb1ac115a06dc966&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:07.176707 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2033c-6555-4369-9581-4ac961d8cf55 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:07.176995 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2033c-6555-4369-9581-4ac961d8cf55 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:07.177323 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2033c-6555-4369-9581-4ac961d8cf55 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:07.179096 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 188/383] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:07.187022 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685b114-7134-4f27-ba10-0e1c49e9c859 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:07.187602 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685b114-7134-4f27-ba10-0e1c49e9c859 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:07.187919 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b685b114-7134-4f27-ba10-0e1c49e9c859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:07.210746 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad62604-e015-4bd3-a330-d05dd77778c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:07.211409 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad62604-e015-4bd3-a330-d05dd77778c7 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:07.211689 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad62604-e015-4bd3-a330-d05dd77778c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:07.211960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad62604-e015-4bd3-a330-d05dd77778c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:07.214683 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-b685b114-7134-4f27-ba10-0e1c49e9c859 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:07.216467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-b685b114-7134-4f27-ba10-0e1c49e9c859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:07.227102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad62604-e015-4bd3-a330-d05dd77778c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:07.232242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-5ad62604-e015-4bd3-a330-d05dd77778c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:40:07.232242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:07.232242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:07.232242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:07.232242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:07.232242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:07.232242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:07.232242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:07.232242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:07.232242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:07.232242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:07.232242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:07.232242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:07.232242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:07.232242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:07.232242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:07.233156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:07.233156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:07.233156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:07.233156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:07.233156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:07.233156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:07.233156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:07.233156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:07.233156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:07.233156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:07.233156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:07.233156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:07.233156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:07.233156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:07.233156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:07.233156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:07.234747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:07.234747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:07.234747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:07.234747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:07.234747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:07.234747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:07.234747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:07.234747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:07.234747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:07.234747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:07.234747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:40:07.234747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:07.234747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 196/384] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:07.239413 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad62604-e015-4bd3-a330-d05dd77778c7 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:07.309444 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 189/385] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:07.335992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685b114-7134-4f27-ba10-0e1c49e9c859 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:07.381215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:07.389488 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad62604-e015-4bd3-a330-d05dd77778c7 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:07.393628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:07.440923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46844176-686c-4ab7-948f-4b3527619e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:07.441536 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46844176-686c-4ab7-948f-4b3527619e09 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:07.441793 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46844176-686c-4ab7-948f-4b3527619e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:07.442070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46844176-686c-4ab7-948f-4b3527619e09 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:07.444403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46844176-686c-4ab7-948f-4b3527619e09 None admin] RBAC: Adding query filter params (name=member) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:07.444403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46844176-686c-4ab7-948f-4b3527619e09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:07.444573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46844176-686c-4ab7-948f-4b3527619e09 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:07.456023 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 197/386] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:40:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:07.462212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46844176-686c-4ab7-948f-4b3527619e09 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:07.492807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b685b114-7134-4f27-ba10-0e1c49e9c859 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:07.541029 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dac5be-a3ba-4d40-92e4-c8e61a925a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:07.541683 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dac5be-a3ba-4d40-92e4-c8e61a925a06 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:07.541987 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dac5be-a3ba-4d40-92e4-c8e61a925a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:07.542289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dac5be-a3ba-4d40-92e4-c8e61a925a06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:07.556753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dac5be-a3ba-4d40-92e4-c8e61a925a06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:07.561967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-07dac5be-a3ba-4d40-92e4-c8e61a925a06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:40:07.561967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:07.561967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:07.561967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:07.561967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:07.561967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:07.561967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:07.561967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:07.561967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:07.561967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:07.561967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:07.561967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:07.561967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:07.561967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:07.561967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:07.561967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:07.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:07.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:07.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:07.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:07.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:07.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:07.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:07.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:07.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:07.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:07.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:07.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:07.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:07.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:07.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:07.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:07.564472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:07.564472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:07.564472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:07.564472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:07.564472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:07.564472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:07.564472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:07.564472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:07.564472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:07.564472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:07.564472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:40:07.564472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:07.564472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 190/387] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:40:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:07.569526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dac5be-a3ba-4d40-92e4-c8e61a925a06 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:07.604025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46844176-686c-4ab7-948f-4b3527619e09 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:07.646196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc530db4-f4ec-4118-ab16-3a06e0cd1606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:07.647261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc530db4-f4ec-4118-ab16-3a06e0cd1606 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:07.647680 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc530db4-f4ec-4118-ab16-3a06e0cd1606 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:07.648115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc530db4-f4ec-4118-ab16-3a06e0cd1606 None admin] PATH_INFO: `/v3/groups/a213174fbe0548ffa7eca114dee63eb7` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:07.649953 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc530db4-f4ec-4118-ab16-3a06e0cd1606 None admin] RBAC: Authorizing `identity:get_group(group_id=a213174fbe0548ffa7eca114dee63eb7)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:07.652010 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc530db4-f4ec-4118-ab16-3a06e0cd1606 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:07.654386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 198/388] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 3 12:40:07 2022] GET /identity/v3/groups/a213174fbe0548ffa7eca114dee63eb7 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:07.661734 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc530db4-f4ec-4118-ab16-3a06e0cd1606 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:07.679403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:07.699378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dac5be-a3ba-4d40-92e4-c8e61a925a06 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:07.709678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:07.742764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc571db-f01e-411a-9fec-c53f77fb5766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:07.743804 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc571db-f01e-411a-9fec-c53f77fb5766 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:07.744288 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc571db-f01e-411a-9fec-c53f77fb5766 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:07.744806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc571db-f01e-411a-9fec-c53f77fb5766 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:07.746116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc571db-f01e-411a-9fec-c53f77fb5766 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:07.746667 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc571db-f01e-411a-9fec-c53f77fb5766 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:07.748260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc571db-f01e-411a-9fec-c53f77fb5766 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:07.758086 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 191/389] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:07.764086 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc571db-f01e-411a-9fec-c53f77fb5766 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:07.789108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc530db4-f4ec-4118-ab16-3a06e0cd1606 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:07.830110 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d4d88-8aaa-461e-a721-04a44c0b4cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:07.830936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d4d88-8aaa-461e-a721-04a44c0b4cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:07.832128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d4d88-8aaa-461e-a721-04a44c0b4cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:07.832520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d4d88-8aaa-461e-a721-04a44c0b4cc1 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:07.833813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d4d88-8aaa-461e-a721-04a44c0b4cc1 None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:07.835371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d4d88-8aaa-461e-a721-04a44c0b4cc1 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:07.838457 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 199/390] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:40:07 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:07.847443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d4d88-8aaa-461e-a721-04a44c0b4cc1 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:07.925053 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc571db-f01e-411a-9fec-c53f77fb5766 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:07.968429 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc4b2d3-669c-46d2-979a-e0acd6a9061b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:07.969483 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4b2d3-669c-46d2-979a-e0acd6a9061b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:07.969971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4b2d3-669c-46d2-979a-e0acd6a9061b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:07.970467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4b2d3-669c-46d2-979a-e0acd6a9061b None admin] PATH_INFO: `/v3/groups/a213174fbe0548ffa7eca114dee63eb7` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:07.972005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4b2d3-669c-46d2-979a-e0acd6a9061b None admin] RBAC: Authorizing `identity:get_group(group_id=a213174fbe0548ffa7eca114dee63eb7)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:07.973930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc4b2d3-669c-46d2-979a-e0acd6a9061b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:07.975823 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 192/391] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 3 12:40:07 2022] GET /identity/v3/groups/a213174fbe0548ffa7eca114dee63eb7 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:07.981456 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc4b2d3-669c-46d2-979a-e0acd6a9061b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:08.004607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:08.054132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d4d88-8aaa-461e-a721-04a44c0b4cc1 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:08.059997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:08.113226 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44248f82-67ef-4160-be64-9a31a2a0f560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:08.114324 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44248f82-67ef-4160-be64-9a31a2a0f560 None admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:08.114870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44248f82-67ef-4160-be64-9a31a2a0f560 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:08.115346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44248f82-67ef-4160-be64-9a31a2a0f560 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119/groups/a213174fbe0548ffa7eca114dee63eb7/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:08.117574 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44248f82-67ef-4160-be64-9a31a2a0f560 None admin] RBAC: Authorizing `identity:create_grant(project_id=22d0a79f59184a36a955aa3a842c3119, group_id=a213174fbe0548ffa7eca114dee63eb7, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:08.119255 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44248f82-67ef-4160-be64-9a31a2a0f560 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:08.143356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc4b2d3-669c-46d2-979a-e0acd6a9061b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:08.146424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 200/392] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 3 12:40:07 2022] PUT /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119/groups/a213174fbe0548ffa7eca114dee63eb7/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:40:08.184271 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3607d60-5d4b-4474-9378-248073a7fd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:08.185182 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3607d60-5d4b-4474-9378-248073a7fd6b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:08.185464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3607d60-5d4b-4474-9378-248073a7fd6b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:08.185741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3607d60-5d4b-4474-9378-248073a7fd6b None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:08.186967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3607d60-5d4b-4474-9378-248073a7fd6b None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:08.188490 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3607d60-5d4b-4474-9378-248073a7fd6b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:08.196147 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44248f82-67ef-4160-be64-9a31a2a0f560 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:08.197838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 193/393] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:40:07 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:08.351533 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44248f82-67ef-4160-be64-9a31a2a0f560 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:08.392121 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1868acf-18f1-4873-8da0-64c87d936818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:08.393160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1868acf-18f1-4873-8da0-64c87d936818 None admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:08.393647 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1868acf-18f1-4873-8da0-64c87d936818 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:08.394123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1868acf-18f1-4873-8da0-64c87d936818 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119/groups/a213174fbe0548ffa7eca114dee63eb7/roles/79ff845a41e241229433560e2b0d6fe5` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:08.396671 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1868acf-18f1-4873-8da0-64c87d936818 None admin] RBAC: Authorizing `identity:create_grant(project_id=22d0a79f59184a36a955aa3a842c3119, group_id=a213174fbe0548ffa7eca114dee63eb7, role_id=79ff845a41e241229433560e2b0d6fe5)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:08.398334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1868acf-18f1-4873-8da0-64c87d936818 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:08.417453 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 201/394] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 3 12:40:08 2022] PUT /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119/groups/a213174fbe0548ffa7eca114dee63eb7/roles/79ff845a41e241229433560e2b0d6fe5 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:40:08.655072 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d554518-7203-43ea-b515-8823c38c5b81 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:08.656091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d554518-7203-43ea-b515-8823c38c5b81 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:08.656375 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d554518-7203-43ea-b515-8823c38c5b81 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:08.658476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 194/395] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:08.664111 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468c9f6-1176-48e8-b3f0-1df30a433ea0 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:08.664952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468c9f6-1176-48e8-b3f0-1df30a433ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:08.665429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468c9f6-1176-48e8-b3f0-1df30a433ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:08.684407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-b468c9f6-1176-48e8-b3f0-1df30a433ea0 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:08.686183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b468c9f6-1176-48e8-b3f0-1df30a433ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:08.704718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:08.761719 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 202/396] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:08.785083 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d554518-7203-43ea-b515-8823c38c5b81 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:08.882165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:08.901844 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d554518-7203-43ea-b515-8823c38c5b81 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:08.938616 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fba10-7b29-46ed-abc3-f9d86962eaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:08.939191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fba10-7b29-46ed-abc3-f9d86962eaa9 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:08.939470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fba10-7b29-46ed-abc3-f9d86962eaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:08.939751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7fba10-7b29-46ed-abc3-f9d86962eaa9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:08.950697 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7fba10-7b29-46ed-abc3-f9d86962eaa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:08.956631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-2e7fba10-7b29-46ed-abc3-f9d86962eaa9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:40:08.956631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:08.956631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:08.956631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:08.956631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:08.956631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:08.956631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:08.956631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:08.956631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:08.956631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:08.956631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:08.956631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:08.956631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:08.956631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:08.956631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:08.956631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:08.957573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:08.957573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:08.957573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:08.957573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:08.957573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:08.957573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:08.957573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:08.957573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:08.957573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:08.957573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:08.957573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:08.957573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:08.957573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:08.957573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:08.957573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:08.957573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:08.958434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:08.958434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:08.958434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:08.958434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:08.958434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:08.958434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:08.958434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:08.958434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:08.958434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:08.958434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:08.958434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:40:08.958434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:08.958434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 195/397] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:08.964393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468c9f6-1176-48e8-b3f0-1df30a433ea0 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:09.071316 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468c9f6-1176-48e8-b3f0-1df30a433ea0 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:09.108420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f370f70-12fc-4d55-8f54-a56160b8e384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:09.109047 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f370f70-12fc-4d55-8f54-a56160b8e384 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:09.109326 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f370f70-12fc-4d55-8f54-a56160b8e384 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:09.109600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f370f70-12fc-4d55-8f54-a56160b8e384 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:09.111047 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f370f70-12fc-4d55-8f54-a56160b8e384 None admin] RBAC: Adding query filter params (name=member) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:09.111355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f370f70-12fc-4d55-8f54-a56160b8e384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:09.112806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f370f70-12fc-4d55-8f54-a56160b8e384 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:09.121982 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 203/398] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:40:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:09.127565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fba10-7b29-46ed-abc3-f9d86962eaa9 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:09.237290 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7fba10-7b29-46ed-abc3-f9d86962eaa9 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:09.272737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253e2d8-77bb-444f-b9a4-29d138188215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:09.273375 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253e2d8-77bb-444f-b9a4-29d138188215 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:09.273656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253e2d8-77bb-444f-b9a4-29d138188215 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:09.273925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253e2d8-77bb-444f-b9a4-29d138188215 None admin] PATH_INFO: `/v3/groups/a213174fbe0548ffa7eca114dee63eb7` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:09.275114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253e2d8-77bb-444f-b9a4-29d138188215 None admin] RBAC: Authorizing `identity:get_group(group_id=a213174fbe0548ffa7eca114dee63eb7)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:09.276551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253e2d8-77bb-444f-b9a4-29d138188215 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:09.278497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 196/399] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 3 12:40:09 2022] GET /identity/v3/groups/a213174fbe0548ffa7eca114dee63eb7 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:09.284567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f370f70-12fc-4d55-8f54-a56160b8e384 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:09.301523 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:09.393734 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f370f70-12fc-4d55-8f54-a56160b8e384 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:09.429902 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dda97d-f3df-4148-b957-04c4b652d0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:09.430548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dda97d-f3df-4148-b957-04c4b652d0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:09.430832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dda97d-f3df-4148-b957-04c4b652d0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:09.431106 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dda97d-f3df-4148-b957-04c4b652d0b9 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:09.432303 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dda97d-f3df-4148-b957-04c4b652d0b9 None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:09.433751 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dda97d-f3df-4148-b957-04c4b652d0b9 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:09.436018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 204/400] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:40:09 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:09.441814 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253e2d8-77bb-444f-b9a4-29d138188215 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:09.458461 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:09.551536 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253e2d8-77bb-444f-b9a4-29d138188215 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:09.553858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7079d-fccd-483f-9515-f9934e285083 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:09.554142 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7079d-fccd-483f-9515-f9934e285083 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:09.554415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7079d-fccd-483f-9515-f9934e285083 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:09.555789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 205/401] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:40:09.564560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ba0e6-4d2c-499b-8cc9-e374278ff4e0 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:09.564560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ba0e6-4d2c-499b-8cc9-e374278ff4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:09.564560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ba0e6-4d2c-499b-8cc9-e374278ff4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:09.583031 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-406ba0e6-4d2c-499b-8cc9-e374278ff4e0 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:09.584535 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-406ba0e6-4d2c-499b-8cc9-e374278ff4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:09.596415 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabd055-20ec-4ece-b455-47a3c79c0cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:09.597119 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabd055-20ec-4ece-b455-47a3c79c0cee None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:09.597442 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabd055-20ec-4ece-b455-47a3c79c0cee None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:09.598404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabd055-20ec-4ece-b455-47a3c79c0cee None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400/groups/a213174fbe0548ffa7eca114dee63eb7/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:09.601008 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabd055-20ec-4ece-b455-47a3c79c0cee None admin] RBAC: Authorizing `identity:create_grant(project_id=efec6052310d4c7289f0ce48ac293400, group_id=a213174fbe0548ffa7eca114dee63eb7, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:09.602910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabd055-20ec-4ece-b455-47a3c79c0cee None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:09.622818 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 197/402] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 3 12:40:09 2022] PUT /identity/v3/projects/efec6052310d4c7289f0ce48ac293400/groups/a213174fbe0548ffa7eca114dee63eb7/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:40:09.681530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 206/403] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:09.707979 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabd055-20ec-4ece-b455-47a3c79c0cee None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:09.794768 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:09.832340 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabd055-20ec-4ece-b455-47a3c79c0cee None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:09.880557 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b05e80-7520-4f48-82a4-91b989d1381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:09.881222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b05e80-7520-4f48-82a4-91b989d1381d None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:09.881506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b05e80-7520-4f48-82a4-91b989d1381d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:09.881784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b05e80-7520-4f48-82a4-91b989d1381d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:09.894777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b05e80-7520-4f48-82a4-91b989d1381d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:09.900306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-58b05e80-7520-4f48-82a4-91b989d1381d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:40:09.900306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:09.900306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:09.900306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:09.900306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:09.900306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:09.900306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:09.900306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:09.900306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:09.900306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:09.900306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:09.900306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:09.900306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:09.900306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:09.900306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:09.900306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:09.901219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:09.901219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:09.901219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:09.901219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:09.901219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:09.901219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:09.901219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:09.901219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:09.901219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:09.901219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:09.901219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:09.901219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:09.901219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:09.901219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:09.901219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:09.901219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:09.902049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:09.902049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:09.902049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:09.902049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:09.902049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:09.902049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:09.902049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:09.902049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:09.902049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:09.902049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:09.902049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:40:09.902049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:09.902049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 198/404] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:40:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:09.907855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ba0e6-4d2c-499b-8cc9-e374278ff4e0 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:09.922790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:10.012322 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ba0e6-4d2c-499b-8cc9-e374278ff4e0 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:10.049578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f03b50-cd58-4ac6-b007-c111d059d615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:10.050213 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f03b50-cd58-4ac6-b007-c111d059d615 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:10.050507 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f03b50-cd58-4ac6-b007-c111d059d615 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:10.050795 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f03b50-cd58-4ac6-b007-c111d059d615 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:10.051610 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f03b50-cd58-4ac6-b007-c111d059d615 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:10.051921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f03b50-cd58-4ac6-b007-c111d059d615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:10.053287 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f03b50-cd58-4ac6-b007-c111d059d615 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:10.062364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 207/405] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:10.068606 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b05e80-7520-4f48-82a4-91b989d1381d None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:10.185534 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b05e80-7520-4f48-82a4-91b989d1381d None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:10.220664 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:10.226895 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87c33a-6ea7-4013-a30b-c7a5fe7b4b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:10.227604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87c33a-6ea7-4013-a30b-c7a5fe7b4b24 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:10.227893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87c33a-6ea7-4013-a30b-c7a5fe7b4b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:10.228171 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87c33a-6ea7-4013-a30b-c7a5fe7b4b24 None admin] PATH_INFO: `/v3/groups/a213174fbe0548ffa7eca114dee63eb7` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:10.231938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87c33a-6ea7-4013-a30b-c7a5fe7b4b24 None admin] RBAC: Authorizing `identity:get_group(group_id=a213174fbe0548ffa7eca114dee63eb7)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:10.233497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87c33a-6ea7-4013-a30b-c7a5fe7b4b24 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:10.235706 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 199/406] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 3 12:40:10 2022] GET /identity/v3/groups/a213174fbe0548ffa7eca114dee63eb7 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:10.241200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f03b50-cd58-4ac6-b007-c111d059d615 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:10.292499 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97568eb-9d2f-4483-a04a-2abaf0aa6670 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:10.292840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97568eb-9d2f-4483-a04a-2abaf0aa6670 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:10.293045 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97568eb-9d2f-4483-a04a-2abaf0aa6670 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:10.294558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 200/407] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:10.300302 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d09b77-01bf-4b27-8ae4-8c47175cf399 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:10.300877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d09b77-01bf-4b27-8ae4-8c47175cf399 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:10.301366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d09b77-01bf-4b27-8ae4-8c47175cf399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:10.328566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-06d09b77-01bf-4b27-8ae4-8c47175cf399 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:10.329783 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-06d09b77-01bf-4b27-8ae4-8c47175cf399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:10.350860 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:10.396421 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f03b50-cd58-4ac6-b007-c111d059d615 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:10.463057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa094fe0-be16-4d12-a1a6-52a56ee23e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:10.463792 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa094fe0-be16-4d12-a1a6-52a56ee23e30 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:10.464080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa094fe0-be16-4d12-a1a6-52a56ee23e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:10.464360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa094fe0-be16-4d12-a1a6-52a56ee23e30 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:10.465195 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 201/408] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:10.465879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa094fe0-be16-4d12-a1a6-52a56ee23e30 None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:10.467396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa094fe0-be16-4d12-a1a6-52a56ee23e30 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:10.469393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 208/409] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:40:10 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:10.475157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d09b77-01bf-4b27-8ae4-8c47175cf399 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:10.499129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa094fe0-be16-4d12-a1a6-52a56ee23e30 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:10.654088 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa094fe0-be16-4d12-a1a6-52a56ee23e30 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:10.671986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d09b77-01bf-4b27-8ae4-8c47175cf399 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:10.713494 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:10.723132 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892842a-4a08-44b3-bce1-c3cbaa604b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:10.724110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892842a-4a08-44b3-bce1-c3cbaa604b15 None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:10.724630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892842a-4a08-44b3-bce1-c3cbaa604b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:10.725115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892842a-4a08-44b3-bce1-c3cbaa604b15 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400/groups/a213174fbe0548ffa7eca114dee63eb7/roles/79ff845a41e241229433560e2b0d6fe5` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:10.727726 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892842a-4a08-44b3-bce1-c3cbaa604b15 None admin] RBAC: Authorizing `identity:create_grant(project_id=efec6052310d4c7289f0ce48ac293400, group_id=a213174fbe0548ffa7eca114dee63eb7, role_id=79ff845a41e241229433560e2b0d6fe5)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:10.729394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892842a-4a08-44b3-bce1-c3cbaa604b15 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:10.748584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 202/410] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 3 12:40:10 2022] PUT /identity/v3/projects/efec6052310d4c7289f0ce48ac293400/groups/a213174fbe0548ffa7eca114dee63eb7/roles/79ff845a41e241229433560e2b0d6fe5 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:40:10.750032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c3b2c-2f58-4b9f-aa16-984519f27602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:10.751819 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c3b2c-2f58-4b9f-aa16-984519f27602 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:10.752307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c3b2c-2f58-4b9f-aa16-984519f27602 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:10.752805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c3b2c-2f58-4b9f-aa16-984519f27602 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:10.765387 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c3b2c-2f58-4b9f-aa16-984519f27602 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:10.772103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-ce6c3b2c-2f58-4b9f-aa16-984519f27602 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:10.772103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:10.772103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:10.772103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:10.772103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:10.772103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:10.772103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:10.772103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:10.772103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:10.772103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:10.772103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:10.772103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:10.772103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:10.772103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:10.772103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:10.772103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:10.773052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:10.773052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:10.773052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:10.773052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:10.773052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:10.773052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:10.773052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:10.773052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:10.773052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:10.773052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:10.773052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:10.773052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:10.773052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:10.773052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:10.773052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:10.773052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:10.773867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:10.773867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:10.773867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:10.773867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:10.773867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:10.773867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:10.773867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:10.773867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:10.773867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:10.773867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:10.773867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:10.773867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:10.776172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 209/411] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:40:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:10.782895 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892842a-4a08-44b3-bce1-c3cbaa604b15 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:10.824502 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bf163-9ebf-4526-82d8-f6f355d77377 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:10.824838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bf163-9ebf-4526-82d8-f6f355d77377 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:10.825099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bf163-9ebf-4526-82d8-f6f355d77377 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:10.827674 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 210/412] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:40:10.832126 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9f880-128d-48b2-b254-502b8df762f2 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:10.832413 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9f880-128d-48b2-b254-502b8df762f2 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:10.832685 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9f880-128d-48b2-b254-502b8df762f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:10.839313 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:10.853827 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-38c9f880-128d-48b2-b254-502b8df762f2 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:10.856389 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-38c9f880-128d-48b2-b254-502b8df762f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:10.921056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892842a-4a08-44b3-bce1-c3cbaa604b15 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:10.955383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 211/413] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:40:10.964854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8472aa-2d03-4e41-8b27-c7fd748b3207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:10.965862 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8472aa-2d03-4e41-8b27-c7fd748b3207 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:10.966350 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8472aa-2d03-4e41-8b27-c7fd748b3207 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:10.966885 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8472aa-2d03-4e41-8b27-c7fd748b3207 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:10.967895 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8472aa-2d03-4e41-8b27-c7fd748b3207 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:10.968408 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8472aa-2d03-4e41-8b27-c7fd748b3207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:10.970165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8472aa-2d03-4e41-8b27-c7fd748b3207 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:10.980538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 203/414] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:10.982369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9f880-128d-48b2-b254-502b8df762f2 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:10.989749 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8472aa-2d03-4e41-8b27-c7fd748b3207 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:11.076373 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:11.119085 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9f880-128d-48b2-b254-502b8df762f2 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:11.135154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8472aa-2d03-4e41-8b27-c7fd748b3207 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:11.167144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:11.176077 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5505ed3-2c8c-4dbc-bc95-ef70d544cf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:11.177299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5505ed3-2c8c-4dbc-bc95-ef70d544cf31 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:11.178044 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5505ed3-2c8c-4dbc-bc95-ef70d544cf31 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:11.178971 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5505ed3-2c8c-4dbc-bc95-ef70d544cf31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:11.184380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dd111-a086-424d-80ae-f339a5dc589c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:11.185282 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dd111-a086-424d-80ae-f339a5dc589c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:11.185564 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dd111-a086-424d-80ae-f339a5dc589c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:11.185837 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dd111-a086-424d-80ae-f339a5dc589c None admin] PATH_INFO: `/v3/groups/1e40589035964a398acba1dd30ca3338` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:11.187117 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8dd111-a086-424d-80ae-f339a5dc589c None admin] RBAC: Authorizing `identity:get_group(group_id=1e40589035964a398acba1dd30ca3338)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:11.188601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8dd111-a086-424d-80ae-f339a5dc589c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:11.190532 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 204/415] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 3 12:40:10 2022] GET /identity/v3/groups/1e40589035964a398acba1dd30ca3338 => generated 229 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:11.193511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5505ed3-2c8c-4dbc-bc95-ef70d544cf31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:11.196324 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dd111-a086-424d-80ae-f339a5dc589c None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:11.199299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-a5505ed3-2c8c-4dbc-bc95-ef70d544cf31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:40:11.199299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:11.199299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:11.199299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:11.199299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:11.199299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:11.199299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:11.199299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:11.199299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:11.199299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:11.199299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:11.199299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:11.199299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:11.199299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:11.199299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:11.199299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:11.200183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:11.200183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:11.200183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:11.200183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:11.200183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:11.200183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:11.200183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:11.200183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:11.200183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:11.200183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:11.200183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:11.200183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:11.200183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:11.200183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:11.200183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:11.200183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:11.200980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:11.200980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:11.200980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:11.200980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:11.200980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:11.200980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:11.200980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:11.200980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:11.200980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:11.200980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:11.200980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:40:11.200980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:11.203876 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 212/416] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:11.210427 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5505ed3-2c8c-4dbc-bc95-ef70d544cf31 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:11.334836 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5505ed3-2c8c-4dbc-bc95-ef70d544cf31 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:11.377611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85630d-5e22-4f07-8589-cf5b82587dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:11.378626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85630d-5e22-4f07-8589-cf5b82587dec None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:11.379745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85630d-5e22-4f07-8589-cf5b82587dec None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:11.380226 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85630d-5e22-4f07-8589-cf5b82587dec None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:11.381336 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85630d-5e22-4f07-8589-cf5b82587dec None admin] RBAC: Adding query filter params (name=member) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:11.381894 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85630d-5e22-4f07-8589-cf5b82587dec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:11.383522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b85630d-5e22-4f07-8589-cf5b82587dec None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:11.394212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 213/417] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:40:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:11.408140 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85630d-5e22-4f07-8589-cf5b82587dec None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:11.431265 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dd111-a086-424d-80ae-f339a5dc589c None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:11.452408 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:11.481586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e33c05-a10b-4b17-ad83-2f06003742e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:11.482318 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e33c05-a10b-4b17-ad83-2f06003742e7 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:11.482632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e33c05-a10b-4b17-ad83-2f06003742e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:11.482906 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e33c05-a10b-4b17-ad83-2f06003742e7 None admin] PATH_INFO: `/v3/projects/87eaffc3c69e48b2bb1ac115a06dc966` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:11.484121 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e33c05-a10b-4b17-ad83-2f06003742e7 None admin] RBAC: Authorizing `identity:get_project(project_id=87eaffc3c69e48b2bb1ac115a06dc966)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:11.486243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e33c05-a10b-4b17-ad83-2f06003742e7 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:11.488062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 205/418] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:40:11 2022] GET /identity/v3/projects/87eaffc3c69e48b2bb1ac115a06dc966 => generated 344 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:11.489955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:11.493738 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e33c05-a10b-4b17-ad83-2f06003742e7 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:11.539956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b85630d-5e22-4f07-8589-cf5b82587dec None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:11.577099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ebbc9-46f7-405c-b66b-a987f62984bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:11.578151 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ebbc9-46f7-405c-b66b-a987f62984bf None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:11.586957 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ebbc9-46f7-405c-b66b-a987f62984bf None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:11.587189 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741ebbc9-46f7-405c-b66b-a987f62984bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:11.635061 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e33c05-a10b-4b17-ad83-2f06003742e7 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:11.679434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48769c8d-ab70-47f0-9911-6506485c29b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:11.680396 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48769c8d-ab70-47f0-9911-6506485c29b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:11.680686 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48769c8d-ab70-47f0-9911-6506485c29b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:11.681200 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48769c8d-ab70-47f0-9911-6506485c29b2 None admin] PATH_INFO: `/v3/projects/87eaffc3c69e48b2bb1ac115a06dc966/groups/1e40589035964a398acba1dd30ca3338/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:11.683874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48769c8d-ab70-47f0-9911-6506485c29b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=87eaffc3c69e48b2bb1ac115a06dc966, group_id=1e40589035964a398acba1dd30ca3338, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:11.686554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48769c8d-ab70-47f0-9911-6506485c29b2 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:11.692614 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ebbc9-46f7-405c-b66b-a987f62984bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:11.694453 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741ebbc9-46f7-405c-b66b-a987f62984bf None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:11.710532 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 206/419] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 3 12:40:11 2022] PUT /identity/v3/projects/87eaffc3c69e48b2bb1ac115a06dc966/groups/1e40589035964a398acba1dd30ca3338/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:40:11.710532 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe24b9c-bdf4-4f2b-b5d0-5ad1619fa964 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:11.710532 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe24b9c-bdf4-4f2b-b5d0-5ad1619fa964 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:11.710532 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe24b9c-bdf4-4f2b-b5d0-5ad1619fa964 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:11.710532 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 207/420] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:11.724290 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d94a4-56e2-499d-b774-8b65729aa953 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:11.725242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d94a4-56e2-499d-b774-8b65729aa953 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:11.725662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d94a4-56e2-499d-b774-8b65729aa953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:11.762904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-061d94a4-56e2-499d-b774-8b65729aa953 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:11.765617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-061d94a4-56e2-499d-b774-8b65729aa953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:11.805920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:11.861648 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 214/421] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:40:11.867643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ebbc9-46f7-405c-b66b-a987f62984bf None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:11.881266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:11.944737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 208/422] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:11.971628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d94a4-56e2-499d-b774-8b65729aa953 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:11.985732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741ebbc9-46f7-405c-b66b-a987f62984bf None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:12.030339 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484753b-a2de-4e82-9863-80816ff97659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:12.031673 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484753b-a2de-4e82-9863-80816ff97659 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:12.032251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484753b-a2de-4e82-9863-80816ff97659 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:12.032870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484753b-a2de-4e82-9863-80816ff97659 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:12.034858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484753b-a2de-4e82-9863-80816ff97659 None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:12.037013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484753b-a2de-4e82-9863-80816ff97659 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:12.039922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 215/423] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:40:11 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:12.045555 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484753b-a2de-4e82-9863-80816ff97659 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:12.076843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:12.090725 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061d94a4-56e2-499d-b774-8b65729aa953 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:12.126679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b80034-8634-42de-9cbe-3829bccc8b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:12.127296 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b80034-8634-42de-9cbe-3829bccc8b04 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:12.127584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b80034-8634-42de-9cbe-3829bccc8b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:12.127932 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b80034-8634-42de-9cbe-3829bccc8b04 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:12.138910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b80034-8634-42de-9cbe-3829bccc8b04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:12.144662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-f6b80034-8634-42de-9cbe-3829bccc8b04 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:40:12.144662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:12.144662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:12.144662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:12.144662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:12.144662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:12.144662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:12.144662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:12.144662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:12.144662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:12.144662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:12.144662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:12.144662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:12.144662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:12.144662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:12.144662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:12.145585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:12.145585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:12.145585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:12.145585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:12.145585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:12.145585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:12.145585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:12.145585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:12.145585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:12.145585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:12.145585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:12.145585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:12.145585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:12.145585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:12.145585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:12.145585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:12.146398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:12.146398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:12.146398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:12.146398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:12.146398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:12.146398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:12.146398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:12.146398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:12.146398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:12.146398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:12.146398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:40:12.146398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:12.146398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 209/424] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:40:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:12.152311 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b80034-8634-42de-9cbe-3829bccc8b04 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:12.178295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484753b-a2de-4e82-9863-80816ff97659 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:12.202839 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:12.228300 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430f784-c5f4-4da6-bbd9-061c59f0b39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:12.229029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430f784-c5f4-4da6-bbd9-061c59f0b39e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:12.229316 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430f784-c5f4-4da6-bbd9-061c59f0b39e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:12.229591 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430f784-c5f4-4da6-bbd9-061c59f0b39e None admin] PATH_INFO: `/v3/groups/a213174fbe0548ffa7eca114dee63eb7` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:12.235256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430f784-c5f4-4da6-bbd9-061c59f0b39e None admin] RBAC: Authorizing `identity:get_group(group_id=a213174fbe0548ffa7eca114dee63eb7)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:12.236726 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430f784-c5f4-4da6-bbd9-061c59f0b39e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:12.238412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 216/425] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 3 12:40:12 2022] GET /identity/v3/groups/a213174fbe0548ffa7eca114dee63eb7 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:12.244299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430f784-c5f4-4da6-bbd9-061c59f0b39e None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:12.266562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b80034-8634-42de-9cbe-3829bccc8b04 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:12.305534 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ee1a1-d9b9-4476-a45e-628ac73a381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:12.306341 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ee1a1-d9b9-4476-a45e-628ac73a381f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:12.306761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ee1a1-d9b9-4476-a45e-628ac73a381f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:12.307174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ee1a1-d9b9-4476-a45e-628ac73a381f None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:12.308114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ee1a1-d9b9-4476-a45e-628ac73a381f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:12.308529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ee1a1-d9b9-4476-a45e-628ac73a381f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:12.310315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ee1a1-d9b9-4476-a45e-628ac73a381f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:12.320478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 210/426] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:12.326216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ee1a1-d9b9-4476-a45e-628ac73a381f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:12.405955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:12.416397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430f784-c5f4-4da6-bbd9-061c59f0b39e None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:12.441029 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ee1a1-d9b9-4476-a45e-628ac73a381f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:12.454817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:12.463023 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1138ed5b-d52d-4399-9802-26a430dbaea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:12.463725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138ed5b-d52d-4399-9802-26a430dbaea7 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:12.464077 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138ed5b-d52d-4399-9802-26a430dbaea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:12.464424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1138ed5b-d52d-4399-9802-26a430dbaea7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:12.465642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138ed5b-d52d-4399-9802-26a430dbaea7 None admin] RBAC: Adding query filter params (group.id=a213174fbe0548ffa7eca114dee63eb7, role.id=0f74302099d9402ba2a922f36cff56ef, scope.project.id=22d0a79f59184a36a955aa3a842c3119) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:12.466274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138ed5b-d52d-4399-9802-26a430dbaea7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:12.468276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1138ed5b-d52d-4399-9802-26a430dbaea7 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:12.476508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 217/427] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 3 12:40:12 2022] GET /identity/v3/role_assignments?group.id=a213174fbe0548ffa7eca114dee63eb7&scope.project.id=22d0a79f59184a36a955aa3a842c3119&role.id=0f74302099d9402ba2a922f36cff56ef => generated 611 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:12.477933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd664cf-c35a-4cd2-ba97-cf1c3c11dca3 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:12.478221 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd664cf-c35a-4cd2-ba97-cf1c3c11dca3 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:12.478517 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd664cf-c35a-4cd2-ba97-cf1c3c11dca3 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:12.479983 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 218/428] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:12.486705 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bca1b-ad11-4085-b3c3-b572c8a50125 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:12.487031 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bca1b-ad11-4085-b3c3-b572c8a50125 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:12.487263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bca1b-ad11-4085-b3c3-b572c8a50125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:12.491132 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be425a4-8111-42bf-a509-8fbe6bd491fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:12.491859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be425a4-8111-42bf-a509-8fbe6bd491fc None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:12.492147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be425a4-8111-42bf-a509-8fbe6bd491fc None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:12.492425 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be425a4-8111-42bf-a509-8fbe6bd491fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:12.510472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-c58bca1b-ad11-4085-b3c3-b572c8a50125 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:12.511904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c58bca1b-ad11-4085-b3c3-b572c8a50125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:12.598805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 219/429] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:12.617598 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bca1b-ad11-4085-b3c3-b572c8a50125 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:12.678684 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be425a4-8111-42bf-a509-8fbe6bd491fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:12.680432 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be425a4-8111-42bf-a509-8fbe6bd491fc None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:12.726845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:12.748674 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bca1b-ad11-4085-b3c3-b572c8a50125 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:12.786643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a54440-9817-46f0-8a04-0f21865933d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:12.787289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a54440-9817-46f0-8a04-0f21865933d3 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:12.787567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a54440-9817-46f0-8a04-0f21865933d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:12.787837 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a54440-9817-46f0-8a04-0f21865933d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:12.799922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a54440-9817-46f0-8a04-0f21865933d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:12.802412 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:12.806056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-37a54440-9817-46f0-8a04-0f21865933d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:40:12.806056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:12.806056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:12.806056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:12.806056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:12.806056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:12.806056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:12.806056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:12.806056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:12.806056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:12.806056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:12.806056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:12.806056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:12.806056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:12.806056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:12.806056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:12.807261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:12.807261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:12.807261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:12.807261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:12.807261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:12.807261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:12.807261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:12.807261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:12.807261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:12.807261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:12.807261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:12.807261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:12.807261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:12.807261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:12.807261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:12.807261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:12.808419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:12.808419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:12.808419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:12.808419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:12.808419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:12.808419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:12.808419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:12.808419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:12.808419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:12.808419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:12.808419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 03 12:40:12.808419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:12.808419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 220/431] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:12.811405 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 211/431] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:40:12.814253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be425a4-8111-42bf-a509-8fbe6bd491fc None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:12.815063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a54440-9817-46f0-8a04-0f21865933d3 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:12.919718 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be425a4-8111-42bf-a509-8fbe6bd491fc None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:12.926072 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a54440-9817-46f0-8a04-0f21865933d3 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:12.956864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fff3d-da58-4f33-b418-2406e435d7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:12.957406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fff3d-da58-4f33-b418-2406e435d7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:12.957684 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fff3d-da58-4f33-b418-2406e435d7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:12.957956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fff3d-da58-4f33-b418-2406e435d7d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:12.958726 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fff3d-da58-4f33-b418-2406e435d7d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:12.959028 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fff3d-da58-4f33-b418-2406e435d7d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:12.960514 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fff3d-da58-4f33-b418-2406e435d7d7 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:12.963796 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e7a71-4306-497a-9def-65b331a47492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:12.964439 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e7a71-4306-497a-9def-65b331a47492 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:12.964712 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e7a71-4306-497a-9def-65b331a47492 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:12.964980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4e7a71-4306-497a-9def-65b331a47492 None admin] PATH_INFO: `/v3/projects/22d0a79f59184a36a955aa3a842c3119` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:12.966323 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e7a71-4306-497a-9def-65b331a47492 None admin] RBAC: Authorizing `identity:get_project(project_id=22d0a79f59184a36a955aa3a842c3119)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:12.967859 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4e7a71-4306-497a-9def-65b331a47492 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:12.970356 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 221/433] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:40:12 2022] GET /identity/v3/projects/22d0a79f59184a36a955aa3a842c3119 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:12.974381 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 212/433] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:40:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:12.975531 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e7a71-4306-497a-9def-65b331a47492 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:12.975930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fff3d-da58-4f33-b418-2406e435d7d7 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:12.992235 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:13.040859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:13.080828 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fff3d-da58-4f33-b418-2406e435d7d7 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:13.090100 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4e7a71-4306-497a-9def-65b331a47492 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:13.115404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736fa54-aceb-4ebb-8ff0-b75364115ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:13.116026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736fa54-aceb-4ebb-8ff0-b75364115ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:13.116310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736fa54-aceb-4ebb-8ff0-b75364115ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:13.116586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736fa54-aceb-4ebb-8ff0-b75364115ef5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:13.126709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86487b-4ab4-4c9d-97ff-ab437aadc8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:13.127356 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86487b-4ab4-4c9d-97ff-ab437aadc8bf None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:13.127686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86487b-4ab4-4c9d-97ff-ab437aadc8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:13.127965 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86487b-4ab4-4c9d-97ff-ab437aadc8bf None admin] PATH_INFO: `/v3/groups/a213174fbe0548ffa7eca114dee63eb7` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:13.129237 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86487b-4ab4-4c9d-97ff-ab437aadc8bf None admin] RBAC: Authorizing `identity:get_group(group_id=a213174fbe0548ffa7eca114dee63eb7)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:13.130794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86487b-4ab4-4c9d-97ff-ab437aadc8bf None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:13.132821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 222/434] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 3 12:40:12 2022] GET /identity/v3/groups/a213174fbe0548ffa7eca114dee63eb7 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:13.138077 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86487b-4ab4-4c9d-97ff-ab437aadc8bf None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:13.189170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736fa54-aceb-4ebb-8ff0-b75364115ef5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:13.190667 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3736fa54-aceb-4ebb-8ff0-b75364115ef5 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:13.229525 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:13.251904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86487b-4ab4-4c9d-97ff-ab437aadc8bf None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:13.252490 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:13.292395 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4003df-8a36-48de-961d-7ab7555739a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:13.293059 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4003df-8a36-48de-961d-7ab7555739a4 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:13.293345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4003df-8a36-48de-961d-7ab7555739a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:13.293618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4003df-8a36-48de-961d-7ab7555739a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:13.294412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4003df-8a36-48de-961d-7ab7555739a4 None admin] RBAC: Adding query filter params (group.id=a213174fbe0548ffa7eca114dee63eb7, role.id=79ff845a41e241229433560e2b0d6fe5, scope.project.id=22d0a79f59184a36a955aa3a842c3119) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:13.294746 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4003df-8a36-48de-961d-7ab7555739a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:13.295274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 213/435] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 12:40:13.296187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4003df-8a36-48de-961d-7ab7555739a4 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:13.301127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736fa54-aceb-4ebb-8ff0-b75364115ef5 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:13.303408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 223/436] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 3 12:40:13 2022] GET /identity/v3/role_assignments?group.id=a213174fbe0548ffa7eca114dee63eb7&scope.project.id=22d0a79f59184a36a955aa3a842c3119&role.id=79ff845a41e241229433560e2b0d6fe5 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:13.304776 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a92e52-0f5b-4038-98bf-eadbfed28262 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:13.305060 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a92e52-0f5b-4038-98bf-eadbfed28262 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:13.305331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a92e52-0f5b-4038-98bf-eadbfed28262 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:13.306872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 224/437] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:13.321625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f992a2-3635-48e0-bf43-466180709bdc None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:13.321978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f992a2-3635-48e0-bf43-466180709bdc None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:13.322253 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f992a2-3635-48e0-bf43-466180709bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:13.343806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-72f992a2-3635-48e0-bf43-466180709bdc None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:13.345972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-72f992a2-3635-48e0-bf43-466180709bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:13.421639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736fa54-aceb-4ebb-8ff0-b75364115ef5 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:13.427757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 225/438] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:13.454844 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f992a2-3635-48e0-bf43-466180709bdc None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:13.466925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f779ac-18ba-458b-add3-fe0dcd01eb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:13.468064 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f779ac-18ba-458b-add3-fe0dcd01eb87 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:13.468570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f779ac-18ba-458b-add3-fe0dcd01eb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:13.469111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f779ac-18ba-458b-add3-fe0dcd01eb87 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:13.471408 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f779ac-18ba-458b-add3-fe0dcd01eb87 None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:13.473695 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f779ac-18ba-458b-add3-fe0dcd01eb87 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:13.480552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 214/439] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:40:13 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:13.481740 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f779ac-18ba-458b-add3-fe0dcd01eb87 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:13.518966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:13.545316 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:13.577849 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f992a2-3635-48e0-bf43-466180709bdc None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:13.621942 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f779ac-18ba-458b-add3-fe0dcd01eb87 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:13.627642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af817b6-31c5-4fe4-81e9-db0dbd457f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:13.628293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af817b6-31c5-4fe4-81e9-db0dbd457f88 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:13.628571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af817b6-31c5-4fe4-81e9-db0dbd457f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:13.628838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af817b6-31c5-4fe4-81e9-db0dbd457f88 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:13.641569 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af817b6-31c5-4fe4-81e9-db0dbd457f88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:13.646904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-1af817b6-31c5-4fe4-81e9-db0dbd457f88 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:40:13.646904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:13.646904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:13.646904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:13.646904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:13.646904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:13.646904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:13.646904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:13.646904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:13.646904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:13.646904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:13.646904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:13.646904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:13.646904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:13.646904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:13.646904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:13.647814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:13.647814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:13.647814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:13.647814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:13.647814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:13.647814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:13.647814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:13.647814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:13.647814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:13.647814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:13.647814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:13.647814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:13.647814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:13.647814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:13.647814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:13.647814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:13.648625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:13.648625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:13.648625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:13.648625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:13.648625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:13.648625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:13.648625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:13.648625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:13.648625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:13.648625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:13.648625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:40:13.648625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:13.648625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 226/440] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:40:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:13.653886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af817b6-31c5-4fe4-81e9-db0dbd457f88 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:13.665563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f1b8a-5056-4c79-9171-1e3a3bb8efdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:13.666469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f1b8a-5056-4c79-9171-1e3a3bb8efdc None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:13.666974 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f1b8a-5056-4c79-9171-1e3a3bb8efdc None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:13.667421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f1b8a-5056-4c79-9171-1e3a3bb8efdc None admin] PATH_INFO: `/v3/groups/a213174fbe0548ffa7eca114dee63eb7` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:13.668848 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f1b8a-5056-4c79-9171-1e3a3bb8efdc None admin] RBAC: Authorizing `identity:get_group(group_id=a213174fbe0548ffa7eca114dee63eb7)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:13.670507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f1b8a-5056-4c79-9171-1e3a3bb8efdc None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:13.672373 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 215/441] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 3 12:40:13 2022] GET /identity/v3/groups/a213174fbe0548ffa7eca114dee63eb7 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:13.678143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f1b8a-5056-4c79-9171-1e3a3bb8efdc None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:13.783857 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af817b6-31c5-4fe4-81e9-db0dbd457f88 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:13.814761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:13.820558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f1b8a-5056-4c79-9171-1e3a3bb8efdc None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:13.832480 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca2e6a-6c85-4736-b9da-2a713cc2cd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:13.833732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca2e6a-6c85-4736-b9da-2a713cc2cd75 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:13.834180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca2e6a-6c85-4736-b9da-2a713cc2cd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:13.834712 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca2e6a-6c85-4736-b9da-2a713cc2cd75 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:13.835654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca2e6a-6c85-4736-b9da-2a713cc2cd75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:13.836194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca2e6a-6c85-4736-b9da-2a713cc2cd75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:13.837870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca2e6a-6c85-4736-b9da-2a713cc2cd75 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:13.839452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:13.848903 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 227/442] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:13.854406 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca2e6a-6c85-4736-b9da-2a713cc2cd75 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:13.868943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36605e5e-e2a8-4e8f-9481-d010f15e28cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:13.869697 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36605e5e-e2a8-4e8f-9481-d010f15e28cf None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:13.869975 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36605e5e-e2a8-4e8f-9481-d010f15e28cf None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:13.870243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36605e5e-e2a8-4e8f-9481-d010f15e28cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:13.871184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36605e5e-e2a8-4e8f-9481-d010f15e28cf None admin] RBAC: Adding query filter params (group.id=a213174fbe0548ffa7eca114dee63eb7, role.id=0f74302099d9402ba2a922f36cff56ef, scope.project.id=efec6052310d4c7289f0ce48ac293400) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:13.871498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36605e5e-e2a8-4e8f-9481-d010f15e28cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:13.876102 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36605e5e-e2a8-4e8f-9481-d010f15e28cf None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:13.881235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 216/443] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 3 12:40:13 2022] GET /identity/v3/role_assignments?group.id=a213174fbe0548ffa7eca114dee63eb7&scope.project.id=efec6052310d4c7289f0ce48ac293400&role.id=0f74302099d9402ba2a922f36cff56ef => generated 611 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:13.974188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca2e6a-6c85-4736-b9da-2a713cc2cd75 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:14.010361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f1e33-b329-497d-b2a7-99f011fcd671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:14.011032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f1e33-b329-497d-b2a7-99f011fcd671 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:14.011314 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f1e33-b329-497d-b2a7-99f011fcd671 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:14.011586 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f1e33-b329-497d-b2a7-99f011fcd671 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:14.080129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:14.087364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f1e33-b329-497d-b2a7-99f011fcd671 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:14.088823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f1e33-b329-497d-b2a7-99f011fcd671 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:14.192977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 228/444] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 12:40:14.198319 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36605e5e-e2a8-4e8f-9481-d010f15e28cf None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:14.293678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c011e06-2ba3-4df3-b7bd-18cb046c0116 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:14.293972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c011e06-2ba3-4df3-b7bd-18cb046c0116 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:14.294252 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c011e06-2ba3-4df3-b7bd-18cb046c0116 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:14.295633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 229/445] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:14.300238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36605e5e-e2a8-4e8f-9481-d010f15e28cf None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:14.300853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28b1f8-1e4e-4441-8bc5-fe01fc7356f0 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:14.301139 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28b1f8-1e4e-4441-8bc5-fe01fc7356f0 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:14.301418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b28b1f8-1e4e-4441-8bc5-fe01fc7356f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:14.319472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-6b28b1f8-1e4e-4441-8bc5-fe01fc7356f0 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:14.320669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-6b28b1f8-1e4e-4441-8bc5-fe01fc7356f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:14.333990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f004e5-c6c1-47c7-bca5-d290925988a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:14.334601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f004e5-c6c1-47c7-bca5-d290925988a0 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:14.334884 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f004e5-c6c1-47c7-bca5-d290925988a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:14.335159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f004e5-c6c1-47c7-bca5-d290925988a0 None admin] PATH_INFO: `/v3/projects/efec6052310d4c7289f0ce48ac293400` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:14.336326 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f004e5-c6c1-47c7-bca5-d290925988a0 None admin] RBAC: Authorizing `identity:get_project(project_id=efec6052310d4c7289f0ce48ac293400)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:14.337776 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f004e5-c6c1-47c7-bca5-d290925988a0 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:14.339461 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 217/446] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:40:14 2022] GET /identity/v3/projects/efec6052310d4c7289f0ce48ac293400 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:14.345616 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f004e5-c6c1-47c7-bca5-d290925988a0 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:14.375697 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:14.385899 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 230/447] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:40:14.408984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28b1f8-1e4e-4441-8bc5-fe01fc7356f0 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:14.434854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:14.444460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f004e5-c6c1-47c7-bca5-d290925988a0 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:14.477143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8aaab-4ff8-4ce1-b87a-240a1c7333b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:14.477778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8aaab-4ff8-4ce1-b87a-240a1c7333b8 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:14.478064 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8aaab-4ff8-4ce1-b87a-240a1c7333b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:14.478335 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8aaab-4ff8-4ce1-b87a-240a1c7333b8 None admin] PATH_INFO: `/v3/groups/a213174fbe0548ffa7eca114dee63eb7` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:14.479531 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8aaab-4ff8-4ce1-b87a-240a1c7333b8 None admin] RBAC: Authorizing `identity:get_group(group_id=a213174fbe0548ffa7eca114dee63eb7)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:14.480977 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8aaab-4ff8-4ce1-b87a-240a1c7333b8 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:14.482585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 218/448] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 3 12:40:14 2022] GET /identity/v3/groups/a213174fbe0548ffa7eca114dee63eb7 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:14.488908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8aaab-4ff8-4ce1-b87a-240a1c7333b8 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:14.506633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b28b1f8-1e4e-4441-8bc5-fe01fc7356f0 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:14.541576 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f9094-7410-402a-ada1-d3f1cecc7824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:14.542206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f9094-7410-402a-ada1-d3f1cecc7824 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:14.542505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f9094-7410-402a-ada1-d3f1cecc7824 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:14.542791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f9094-7410-402a-ada1-d3f1cecc7824 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:14.553316 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f9094-7410-402a-ada1-d3f1cecc7824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:14.558196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-5c1f9094-7410-402a-ada1-d3f1cecc7824 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:14.558196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:14.558196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:14.558196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:14.558196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:14.558196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:14.558196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:14.558196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:14.558196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:14.558196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:14.558196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:14.558196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:14.558196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:14.558196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:14.558196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:14.558196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:14.559146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:14.559146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:14.559146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:14.559146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:14.559146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:14.559146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:14.559146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:14.559146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:14.559146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:14.559146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:14.559146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:14.559146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:14.559146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:14.559146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:14.559146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:14.559146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:14.560228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:14.560228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:14.560228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:14.560228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:14.560228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:14.560228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:14.560228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:14.560228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:14.560228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:14.560228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:14.560228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:14.560228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:14.560228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 231/449] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:40:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:14.565448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f9094-7410-402a-ada1-d3f1cecc7824 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:14.583569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:14.587599 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8aaab-4ff8-4ce1-b87a-240a1c7333b8 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:14.620711 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390b98a-7b65-4429-96e8-2b4f06b833cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:14.621307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390b98a-7b65-4429-96e8-2b4f06b833cf None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:14.621594 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390b98a-7b65-4429-96e8-2b4f06b833cf None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:14.621866 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390b98a-7b65-4429-96e8-2b4f06b833cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:14.622690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390b98a-7b65-4429-96e8-2b4f06b833cf None admin] RBAC: Adding query filter params (group.id=a213174fbe0548ffa7eca114dee63eb7, role.id=79ff845a41e241229433560e2b0d6fe5, scope.project.id=efec6052310d4c7289f0ce48ac293400) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:14.623029 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390b98a-7b65-4429-96e8-2b4f06b833cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:14.624453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390b98a-7b65-4429-96e8-2b4f06b833cf None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:14.630124 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 219/450] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 3 12:40:14 2022] GET /identity/v3/role_assignments?group.id=a213174fbe0548ffa7eca114dee63eb7&scope.project.id=efec6052310d4c7289f0ce48ac293400&role.id=79ff845a41e241229433560e2b0d6fe5 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:14.652727 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:14.669434 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f9094-7410-402a-ada1-d3f1cecc7824 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:14.705343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc270da8-a110-4624-97d8-9ce280be9f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:14.705967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc270da8-a110-4624-97d8-9ce280be9f68 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:14.706243 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc270da8-a110-4624-97d8-9ce280be9f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:14.706540 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc270da8-a110-4624-97d8-9ce280be9f68 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:14.707275 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc270da8-a110-4624-97d8-9ce280be9f68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:14.707607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc270da8-a110-4624-97d8-9ce280be9f68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:14.708995 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc270da8-a110-4624-97d8-9ce280be9f68 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:14.717897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 232/451] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:14.724318 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390b98a-7b65-4429-96e8-2b4f06b833cf None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:14.827100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390b98a-7b65-4429-96e8-2b4f06b833cf None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:14.863858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322486e-355b-45dc-8463-0c5dd4846374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:14.864420 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322486e-355b-45dc-8463-0c5dd4846374 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:14.864710 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322486e-355b-45dc-8463-0c5dd4846374 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:14.864978 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322486e-355b-45dc-8463-0c5dd4846374 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:14.901142 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:14.940917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322486e-355b-45dc-8463-0c5dd4846374 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:14.942356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322486e-355b-45dc-8463-0c5dd4846374 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:15.037141 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 220/452] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:40:15.042788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc270da8-a110-4624-97d8-9ce280be9f68 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:15.137516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc270da8-a110-4624-97d8-9ce280be9f68 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:15.171558 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d64fb1-61fc-4b8d-a647-b03f389f23f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:15.172207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d64fb1-61fc-4b8d-a647-b03f389f23f9 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:15.172484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d64fb1-61fc-4b8d-a647-b03f389f23f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:15.172756 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d64fb1-61fc-4b8d-a647-b03f389f23f9 None admin] PATH_INFO: `/v3/projects/87eaffc3c69e48b2bb1ac115a06dc966` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:15.173897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d64fb1-61fc-4b8d-a647-b03f389f23f9 None admin] RBAC: Authorizing `identity:get_project(project_id=87eaffc3c69e48b2bb1ac115a06dc966)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:15.175331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d64fb1-61fc-4b8d-a647-b03f389f23f9 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:15.176939 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 233/453] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:40:15 2022] GET /identity/v3/projects/87eaffc3c69e48b2bb1ac115a06dc966 => generated 344 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:15.182580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322486e-355b-45dc-8463-0c5dd4846374 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:15.241047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:15.278904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322486e-355b-45dc-8463-0c5dd4846374 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:15.311560 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc9978-3055-421b-bbf5-a6c6bed98e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:15.312179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc9978-3055-421b-bbf5-a6c6bed98e86 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:15.312456 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc9978-3055-421b-bbf5-a6c6bed98e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:15.313338 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc9978-3055-421b-bbf5-a6c6bed98e86 None admin] PATH_INFO: `/v3/groups/1e40589035964a398acba1dd30ca3338` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:15.314516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc9978-3055-421b-bbf5-a6c6bed98e86 None admin] RBAC: Authorizing `identity:get_group(group_id=1e40589035964a398acba1dd30ca3338)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:15.315929 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc9978-3055-421b-bbf5-a6c6bed98e86 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:15.317527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 221/454] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 3 12:40:15 2022] GET /identity/v3/groups/1e40589035964a398acba1dd30ca3338 => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:15.323230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d64fb1-61fc-4b8d-a647-b03f389f23f9 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:15.340953 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:15.417619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d64fb1-61fc-4b8d-a647-b03f389f23f9 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:15.450214 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99767e-e02a-4fa6-8f64-af6fcece3e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:15.450842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99767e-e02a-4fa6-8f64-af6fcece3e94 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:15.451117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99767e-e02a-4fa6-8f64-af6fcece3e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:15.451382 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99767e-e02a-4fa6-8f64-af6fcece3e94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:15.452156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99767e-e02a-4fa6-8f64-af6fcece3e94 None admin] RBAC: Adding query filter params (group.id=1e40589035964a398acba1dd30ca3338, role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=87eaffc3c69e48b2bb1ac115a06dc966) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:15.452460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99767e-e02a-4fa6-8f64-af6fcece3e94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:15.453908 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99767e-e02a-4fa6-8f64-af6fcece3e94 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:15.460530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 234/455] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 3 12:40:15 2022] GET /identity/v3/role_assignments?group.id=1e40589035964a398acba1dd30ca3338&scope.project.id=87eaffc3c69e48b2bb1ac115a06dc966&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:17.836556 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cdd7f-dd19-41d2-a90a-e67ba97ce66b None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:17.837077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cdd7f-dd19-41d2-a90a-e67ba97ce66b None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:17.837077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cdd7f-dd19-41d2-a90a-e67ba97ce66b None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:17.838705 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 222/456] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:17.843899 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0fd02-ad14-4226-9b2c-b46e81ee8673 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:17.844355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0fd02-ad14-4226-9b2c-b46e81ee8673 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:17.845090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0fd02-ad14-4226-9b2c-b46e81ee8673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:17.871678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-56d0fd02-ad14-4226-9b2c-b46e81ee8673 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:17.872880 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-56d0fd02-ad14-4226-9b2c-b46e81ee8673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:17.956313 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 235/457] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:18.003926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cdd7f-dd19-41d2-a90a-e67ba97ce66b None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:18.040642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb70440-a9b5-4548-a45d-202680e79ae3 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:18.040930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb70440-a9b5-4548-a45d-202680e79ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:18.041183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb70440-a9b5-4548-a45d-202680e79ae3 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:18.042796 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 236/458] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:40:18.053789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305d167-5c73-47d1-af38-d0dedc1b00b9 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:18.054065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305d167-5c73-47d1-af38-d0dedc1b00b9 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:18.054385 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305d167-5c73-47d1-af38-d0dedc1b00b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:18.083948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1305d167-5c73-47d1-af38-d0dedc1b00b9 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:18.085136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1305d167-5c73-47d1-af38-d0dedc1b00b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:18.095556 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:18.121383 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cdd7f-dd19-41d2-a90a-e67ba97ce66b None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:18.186838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:18.192610 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a934b9e-4763-465d-8343-7e76d29d336f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:18.193263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a934b9e-4763-465d-8343-7e76d29d336f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:18.193543 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a934b9e-4763-465d-8343-7e76d29d336f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:18.197054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a934b9e-4763-465d-8343-7e76d29d336f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:18.218638 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a934b9e-4763-465d-8343-7e76d29d336f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:18.220157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a934b9e-4763-465d-8343-7e76d29d336f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:18.225898 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 237/459] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:40:18.248691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-1a934b9e-4763-465d-8343-7e76d29d336f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:18.248691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:18.248691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:18.248691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:18.248691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:18.248691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:18.248691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:18.248691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:18.248691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:18.248691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:18.248691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:18.248691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:18.248691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:18.248691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:18.248691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:18.249618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:18.251049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:18.251049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:18.251049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:18.251049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:18.251049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 223/460] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:18.256225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305d167-5c73-47d1-af38-d0dedc1b00b9 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:18.282575 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a934b9e-4763-465d-8343-7e76d29d336f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:18.377239 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305d167-5c73-47d1-af38-d0dedc1b00b9 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:18.388051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a934b9e-4763-465d-8343-7e76d29d336f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:18.415614 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78291d3-75f3-4c45-91ee-50d1808790da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:18.416345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78291d3-75f3-4c45-91ee-50d1808790da None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:18.416725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78291d3-75f3-4c45-91ee-50d1808790da None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:18.417007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78291d3-75f3-4c45-91ee-50d1808790da None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:18.417711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78291d3-75f3-4c45-91ee-50d1808790da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:18.418015 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78291d3-75f3-4c45-91ee-50d1808790da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:18.419521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78291d3-75f3-4c45-91ee-50d1808790da None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:18.424958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ce508-3525-4be1-8b1a-d02bc52437dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:18.425603 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ce508-3525-4be1-8b1a-d02bc52437dc None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:18.425885 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ce508-3525-4be1-8b1a-d02bc52437dc None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:18.426163 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ce508-3525-4be1-8b1a-d02bc52437dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:18.435130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:18.437544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ce508-3525-4be1-8b1a-d02bc52437dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:18.439792 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ce508-3525-4be1-8b1a-d02bc52437dc None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:18.440888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 238/461] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:18.442578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bc5fe-ff51-496c-83cf-03526311fd97 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:18.443101 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bc5fe-ff51-496c-83cf-03526311fd97 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:18.443534 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bc5fe-ff51-496c-83cf-03526311fd97 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:18.448956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 239/462] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:18.449997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646bc5fe-ff51-496c-83cf-03526311fd97 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:18.455414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-cc8ce508-3525-4be1-8b1a-d02bc52437dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:18.455414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:18.455414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:18.455414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:18.455414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:18.455414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:18.455414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:18.455414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:18.455414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:18.455414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:18.455414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:18.455414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:18.455414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:18.455414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:18.455414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:18.456473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:18.457653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:18.457653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:18.457653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:18.457653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:18.457653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 224/463] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:18.457987 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac402043-46a5-4cb1-a053-389005151203 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:18.458262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac402043-46a5-4cb1-a053-389005151203 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:18.463147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac402043-46a5-4cb1-a053-389005151203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:18.481740 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-ac402043-46a5-4cb1-a053-389005151203 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:18.482953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ac402043-46a5-4cb1-a053-389005151203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:18.512103 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:18.555518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 225/464] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:18.563244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac402043-46a5-4cb1-a053-389005151203 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:18.576078 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646bc5fe-ff51-496c-83cf-03526311fd97 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:18.623968 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e0e8a-207d-4be4-bee5-d7490655953c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:18.624592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e0e8a-207d-4be4-bee5-d7490655953c None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:18.624872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e0e8a-207d-4be4-bee5-d7490655953c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:18.625689 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e0e8a-207d-4be4-bee5-d7490655953c None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:18.626389 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e0e8a-207d-4be4-bee5-d7490655953c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:18.627929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72e0e8a-207d-4be4-bee5-d7490655953c None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:18.650229 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 240/465] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:40:18 2022] POST /identity/v3/users => generated 246 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:40:18.651569 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e0e8a-207d-4be4-bee5-d7490655953c None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:18.665316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac402043-46a5-4cb1-a053-389005151203 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:18.715874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1485192-652c-468c-a262-98729dfe3209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:18.717063 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1485192-652c-468c-a262-98729dfe3209 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:18.717425 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1485192-652c-468c-a262-98729dfe3209 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:18.717772 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1485192-652c-468c-a262-98729dfe3209 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:18.718625 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1485192-652c-468c-a262-98729dfe3209 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:18.719912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1485192-652c-468c-a262-98729dfe3209 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:18.722386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1485192-652c-468c-a262-98729dfe3209 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:18.733381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:18.746147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 226/466] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:18.746147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a5864-e1a9-4ca5-9119-b4e7c78bb319 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:18.746147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a5864-e1a9-4ca5-9119-b4e7c78bb319 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:18.746147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a5864-e1a9-4ca5-9119-b4e7c78bb319 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:18.746147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 227/467] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:18.751570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a5864-e1a9-4ca5-9119-b4e7c78bb319 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:18.787441 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:18.795461 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72e0e8a-207d-4be4-bee5-d7490655953c None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:18.839893 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe51524-6fdd-453b-b6ec-2f1d1f527c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:18.840646 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe51524-6fdd-453b-b6ec-2f1d1f527c91 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:18.840936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe51524-6fdd-453b-b6ec-2f1d1f527c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:18.841231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe51524-6fdd-453b-b6ec-2f1d1f527c91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:18.854219 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe51524-6fdd-453b-b6ec-2f1d1f527c91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:18.855758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe51524-6fdd-453b-b6ec-2f1d1f527c91 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:18.879338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-abe51524-6fdd-453b-b6ec-2f1d1f527c91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:18.879338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:18.879338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:18.879338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:18.879338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:18.879338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:18.879338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:18.879338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:18.879338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:18.879338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:18.879338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:18.879338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:18.879338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:18.879338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:18.879338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:18.880735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:18.881596 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:18.881596 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:18.881596 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:18.881596 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:18.881596 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 241/468] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:18.881870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb0cbd-712a-4d96-bac6-394a1d478247 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:18.883507 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb0cbd-712a-4d96-bac6-394a1d478247 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:18.883507 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb0cbd-712a-4d96-bac6-394a1d478247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:18.900570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9a5864-e1a9-4ca5-9119-b4e7c78bb319 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:18.905263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-64cb0cbd-712a-4d96-bac6-394a1d478247 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:18.907122 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-64cb0cbd-712a-4d96-bac6-394a1d478247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:18.943510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf7f6d-e90e-446c-80d4-f8a4bdf95dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:18.944135 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf7f6d-e90e-446c-80d4-f8a4bdf95dad None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:18.944412 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf7f6d-e90e-446c-80d4-f8a4bdf95dad None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:18.944687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf7f6d-e90e-446c-80d4-f8a4bdf95dad None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:18.945528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf7f6d-e90e-446c-80d4-f8a4bdf95dad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:18.947822 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf7f6d-e90e-446c-80d4-f8a4bdf95dad None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:18.979051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 228/469] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:40:18 2022] POST /identity/v3/users => generated 248 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:40:18.980237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf7f6d-e90e-446c-80d4-f8a4bdf95dad None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:18.982098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 242/470] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:19.067592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb0cbd-712a-4d96-bac6-394a1d478247 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:19.106192 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:19.129804 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf7f6d-e90e-446c-80d4-f8a4bdf95dad None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:19.162906 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:19.174908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c68359-c34d-45d9-a45b-16c02e26086c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:19.175808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c68359-c34d-45d9-a45b-16c02e26086c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:19.176094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c68359-c34d-45d9-a45b-16c02e26086c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:19.176369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c68359-c34d-45d9-a45b-16c02e26086c None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:19.177085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c68359-c34d-45d9-a45b-16c02e26086c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:19.177388 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c68359-c34d-45d9-a45b-16c02e26086c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:19.179447 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c68359-c34d-45d9-a45b-16c02e26086c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:19.196720 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 229/471] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:19.202206 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c68359-c34d-45d9-a45b-16c02e26086c None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:19.237538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cb0cbd-712a-4d96-bac6-394a1d478247 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:19.275852 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0b825-3daf-4fe8-af47-2cf3b9825f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:19.277046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0b825-3daf-4fe8-af47-2cf3b9825f24 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:19.277656 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0b825-3daf-4fe8-af47-2cf3b9825f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:19.278120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0b825-3daf-4fe8-af47-2cf3b9825f24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:19.289604 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0b825-3daf-4fe8-af47-2cf3b9825f24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:19.291443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0b825-3daf-4fe8-af47-2cf3b9825f24 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:19.308617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-dcc0b825-3daf-4fe8-af47-2cf3b9825f24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:19.308617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:19.308617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:19.308617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:19.308617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:19.308617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:19.308617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:19.308617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:19.308617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:19.308617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:19.308617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:19.308617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:19.308617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:19.308617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:19.308617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:19.309794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:19.310695 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:19.310695 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:19.310695 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:19.310695 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:19.310915 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c68359-c34d-45d9-a45b-16c02e26086c None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:19.313696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 243/472] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:19.319587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0b825-3daf-4fe8-af47-2cf3b9825f24 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:19.345902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d2547-aab2-406f-a13e-612043670b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:19.346602 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d2547-aab2-406f-a13e-612043670b99 None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:19.346884 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d2547-aab2-406f-a13e-612043670b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:19.347157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d2547-aab2-406f-a13e-612043670b99 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:19.347855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d2547-aab2-406f-a13e-612043670b99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:19.349305 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d2547-aab2-406f-a13e-612043670b99 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:19.363769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:19.378643 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 230/473] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:40:19 2022] POST /identity/v3/users => generated 248 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:40:19.380687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b304cd-1576-4bac-b13d-7f90f9b487ea None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:19.380944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b304cd-1576-4bac-b13d-7f90f9b487ea None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:19.381479 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b304cd-1576-4bac-b13d-7f90f9b487ea None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:19.384529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 231/474] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:19.391874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e8903-7641-4d06-8614-87f288e1ac76 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:19.392094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e8903-7641-4d06-8614-87f288e1ac76 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:19.392367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018e8903-7641-4d06-8614-87f288e1ac76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:19.415209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-018e8903-7641-4d06-8614-87f288e1ac76 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:19.416582 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-018e8903-7641-4d06-8614-87f288e1ac76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:19.436679 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:19.447923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0b825-3daf-4fe8-af47-2cf3b9825f24 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:19.503257 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921be5a-157c-4595-a2e0-e5140000ef6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:19.503880 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921be5a-157c-4595-a2e0-e5140000ef6f None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:19.504158 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921be5a-157c-4595-a2e0-e5140000ef6f None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:19.504430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921be5a-157c-4595-a2e0-e5140000ef6f None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:19.505138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921be5a-157c-4595-a2e0-e5140000ef6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:19.506220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921be5a-157c-4595-a2e0-e5140000ef6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:19.507652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921be5a-157c-4595-a2e0-e5140000ef6f None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:19.518458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 232/475] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:40:19.535731 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 244/476] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:19.541533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e8903-7641-4d06-8614-87f288e1ac76 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:19.578948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921be5a-157c-4595-a2e0-e5140000ef6f None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:19.651838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018e8903-7641-4d06-8614-87f288e1ac76 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:19.678968 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:19.688849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3934ebb-59c3-4d5a-80a2-99c295d52c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:19.689686 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3934ebb-59c3-4d5a-80a2-99c295d52c45 None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:19.690174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3934ebb-59c3-4d5a-80a2-99c295d52c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:19.690455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3934ebb-59c3-4d5a-80a2-99c295d52c45 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:19.691202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3934ebb-59c3-4d5a-80a2-99c295d52c45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:19.692627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3934ebb-59c3-4d5a-80a2-99c295d52c45 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:19.693252 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921be5a-157c-4595-a2e0-e5140000ef6f None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:19.717301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 233/477] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:40:19 2022] POST /identity/v3/users => generated 249 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:40:19.730977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be196604-bf06-4b9c-91e0-87bcf48bbcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:19.731618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be196604-bf06-4b9c-91e0-87bcf48bbcff None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:19.731909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be196604-bf06-4b9c-91e0-87bcf48bbcff None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:19.732226 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be196604-bf06-4b9c-91e0-87bcf48bbcff None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:19.732966 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be196604-bf06-4b9c-91e0-87bcf48bbcff None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:19.734379 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be196604-bf06-4b9c-91e0-87bcf48bbcff None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:19.744814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.sql.core [None req-be196604-bf06-4b9c-91e0-87bcf48bbcff None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 03 12:40:19.744814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 03 12:40:19.744814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [parameters: {'id': '9df20d296e2248a6bcd9769b72d2e279', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 03 12:40:19.744814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80305) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 03 12:40:19.761853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-be196604-bf06-4b9c-91e0-87bcf48bbcff None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 03 12:40:19.761853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:19.761853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 12:40:19.761853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 12:40:19.761853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 12:40:19.761853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 12:40:19.761853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 12:40:19.761853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 12:40:19.761853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 12:40:19.761853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application conn.query(q) Aug 03 12:40:19.761853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 12:40:19.761853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 12:40:19.761853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 12:40:19.761853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application result.read() Aug 03 12:40:19.761853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return ref.to_dict() Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 12:40:19.763948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application next(self.gen) Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application yield resource Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application next(self.gen) Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self.session.rollback() Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self.force_reraise() Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise self.value Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application session.commit() Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 03 12:40:19.766771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._prepare_impl() Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self.session.flush() Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._flush(objects) Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application compat.raise_( Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application flush_context.execute() Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rec.execute(self) Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 03 12:40:19.769098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 03 12:40:19.771157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 03 12:40:19.771157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 03 12:40:19.771157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 03 12:40:19.771157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 03 12:40:19.771157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 03 12:40:19.771157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 03 12:40:19.771157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 03 12:40:19.771157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 03 12:40:19.771157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 03 12:40:19.771157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 03 12:40:19.771157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 03 12:40:19.771157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 03 12:40:19.771157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 03 12:40:19.771157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 03 12:40:19.771157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 12:40:19.771157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception Aug 03 12:40:19.779119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 12:40:19.779119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 12:40:19.779119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 12:40:19.779119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 12:40:19.779119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 12:40:19.779119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 12:40:19.779119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 12:40:19.779119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application conn.query(q) Aug 03 12:40:19.779119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 12:40:19.779119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 12:40:19.779119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 12:40:19.779119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application result.read() Aug 03 12:40:19.779119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 12:40:19.779119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 12:40:19.779119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 12:40:19.779119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 12:40:19.780117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 12:40:19.780117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 12:40:19.780117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 12:40:19.780117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 12:40:19.780117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 03 12:40:19.780117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 03 12:40:19.780117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [parameters: {'id': '9df20d296e2248a6bcd9769b72d2e279', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 03 12:40:19.780117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 03 12:40:19.780117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:19.780117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:40:19.780117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:19.780117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:19.780117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:19.780117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:19.780117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:19.780117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:19.781188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:19.781188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:19.781188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:19.781188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:19.781188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:19.781188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:19.781188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 03 12:40:19.781188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 03 12:40:19.781188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:19.781188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:19.781188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 03 12:40:19.781188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 03 12:40:19.781188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 03 12:40:19.781188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 03 12:40:19.781188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 03 12:40:19.781188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:19.782921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 245/478] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 3 12:40:19 2022] POST /identity/v3/roles => generated 144 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:19.783000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3934ebb-59c3-4d5a-80a2-99c295d52c45 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:19.882395 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3934ebb-59c3-4d5a-80a2-99c295d52c45 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:19.919425 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68230f60-61a8-4c89-b57c-4fc6bc0a536e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:19.920755 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68230f60-61a8-4c89-b57c-4fc6bc0a536e None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:19.921040 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68230f60-61a8-4c89-b57c-4fc6bc0a536e None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:19.921309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68230f60-61a8-4c89-b57c-4fc6bc0a536e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:19.932538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68230f60-61a8-4c89-b57c-4fc6bc0a536e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:19.938232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-68230f60-61a8-4c89-b57c-4fc6bc0a536e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:40:19.938232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:19.938232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:19.938232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:19.938232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:19.938232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:19.938232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:19.938232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:19.938232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:19.938232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:19.938232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:19.938232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:19.938232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:19.938232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:19.938232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:19.938232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:19.939225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:19.939225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:19.939225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:19.939225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:19.939225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:19.939225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:19.939225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:19.939225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:19.939225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:19.939225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:19.939225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:19.939225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:19.939225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:19.939225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:19.939225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:19.939225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:19.940180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:19.940180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:19.940180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:19.940180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:19.940180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:19.940180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:19.940180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:19.940180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:19.940180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:19.940180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:19.940180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 03 12:40:19.940180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:19.940180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 234/479] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:40:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:19.944997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be196604-bf06-4b9c-91e0-87bcf48bbcff None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:19.965329 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:20.055419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be196604-bf06-4b9c-91e0-87bcf48bbcff None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:20.090218 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cd0ef-44dd-40f0-bc55-ae810aac749e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:20.090894 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cd0ef-44dd-40f0-bc55-ae810aac749e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:20.091170 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cd0ef-44dd-40f0-bc55-ae810aac749e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:20.091442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cd0ef-44dd-40f0-bc55-ae810aac749e None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:20.092162 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cd0ef-44dd-40f0-bc55-ae810aac749e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:20.092461 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cd0ef-44dd-40f0-bc55-ae810aac749e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:20.094438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cd0ef-44dd-40f0-bc55-ae810aac749e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:20.103432 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 246/480] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:21.053143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72deb948-a137-4207-9c0b-f06d24f7487e None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:21.054136 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72deb948-a137-4207-9c0b-f06d24f7487e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:21.054683 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72deb948-a137-4207-9c0b-f06d24f7487e None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:21.056468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 235/481] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:21.062342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fb4cf-9767-405a-b552-725ebe10ea8b None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:21.063676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fb4cf-9767-405a-b552-725ebe10ea8b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:21.064198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fb4cf-9767-405a-b552-725ebe10ea8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:21.086622 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-b69fb4cf-9767-405a-b552-725ebe10ea8b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:21.088114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b69fb4cf-9767-405a-b552-725ebe10ea8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:21.169222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 247/482] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:21.220473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72deb948-a137-4207-9c0b-f06d24f7487e None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:21.280071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:21.363177 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72deb948-a137-4207-9c0b-f06d24f7487e None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:21.407849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943f4ad-a3b9-4f70-9e52-a127a142c6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:21.408835 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943f4ad-a3b9-4f70-9e52-a127a142c6dd None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:21.409310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943f4ad-a3b9-4f70-9e52-a127a142c6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:21.409804 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943f4ad-a3b9-4f70-9e52-a127a142c6dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:21.426448 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943f4ad-a3b9-4f70-9e52-a127a142c6dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:21.431586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-f943f4ad-a3b9-4f70-9e52-a127a142c6dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:21.431586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:21.431586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:21.431586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:21.431586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:21.431586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:21.431586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:21.431586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:21.431586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:21.431586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:21.431586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:21.431586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:21.431586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:21.431586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:21.431586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:21.431586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:21.432469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:21.432469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:21.432469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:21.432469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:21.432469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:21.432469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:21.432469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:21.432469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:21.432469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:21.432469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:21.432469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:21.432469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:21.432469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:21.432469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:21.432469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:21.432469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:21.433260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:21.433260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:21.433260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:21.433260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:21.433260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:21.433260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:21.433260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:21.433260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:21.433260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:21.433260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:21.433260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:21.433260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:21.436148 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 236/483] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:21.441780 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fb4cf-9767-405a-b552-725ebe10ea8b None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:21.458836 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:21.567576 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fb4cf-9767-405a-b552-725ebe10ea8b None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:21.623891 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df18e7c-ca04-44f3-9655-47698ee43160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:21.624531 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df18e7c-ca04-44f3-9655-47698ee43160 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:21.624804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df18e7c-ca04-44f3-9655-47698ee43160 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:21.625110 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df18e7c-ca04-44f3-9655-47698ee43160 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:21.625830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df18e7c-ca04-44f3-9655-47698ee43160 None admin] RBAC: Adding query filter params (name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:21.626187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df18e7c-ca04-44f3-9655-47698ee43160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:21.627591 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df18e7c-ca04-44f3-9655-47698ee43160 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:21.636314 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 248/484] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:21.642435 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943f4ad-a3b9-4f70-9e52-a127a142c6dd None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:21.760455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943f4ad-a3b9-4f70-9e52-a127a142c6dd None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:21.770881 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:21.788758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd521dd-1b96-4274-87c8-e7eef9aaccd4 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:21.789400 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd521dd-1b96-4274-87c8-e7eef9aaccd4 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:21.789906 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd521dd-1b96-4274-87c8-e7eef9aaccd4 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:21.791507 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 249/485] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:40:21.797026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712cd07-11c9-4e23-a78a-10f1546ba6d5 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:21.797624 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712cd07-11c9-4e23-a78a-10f1546ba6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:21.798104 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712cd07-11c9-4e23-a78a-10f1546ba6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:21.811727 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5d6cf-50df-4c1a-8c5b-9065fcb04e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:21.812376 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5d6cf-50df-4c1a-8c5b-9065fcb04e47 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:21.813191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5d6cf-50df-4c1a-8c5b-9065fcb04e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:21.813458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5d6cf-50df-4c1a-8c5b-9065fcb04e47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:21.824000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-0712cd07-11c9-4e23-a78a-10f1546ba6d5 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:21.825635 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0712cd07-11c9-4e23-a78a-10f1546ba6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:21.834256 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d5d6cf-50df-4c1a-8c5b-9065fcb04e47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:21.835736 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d5d6cf-50df-4c1a-8c5b-9065fcb04e47 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:21.858812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-d1d5d6cf-50df-4c1a-8c5b-9065fcb04e47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:21.858812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:21.858812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:21.858812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:21.858812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:21.858812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:21.858812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:21.858812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:21.858812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:21.858812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:21.858812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:21.858812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:21.858812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:21.858812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:21.858812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:21.859778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:21.860624 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:21.860624 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:21.860624 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:21.860624 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:21.867891 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 237/486] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:21.867891 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5d6cf-50df-4c1a-8c5b-9065fcb04e47 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:21.990869 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:22.002740 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5d6cf-50df-4c1a-8c5b-9065fcb04e47 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:22.016749 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 250/487] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:40:22.043215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5920e-2e01-4f57-8da2-d2b7d2c9e05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:22.043894 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5920e-2e01-4f57-8da2-d2b7d2c9e05f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:22.044181 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5920e-2e01-4f57-8da2-d2b7d2c9e05f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:22.044453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5920e-2e01-4f57-8da2-d2b7d2c9e05f None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:22.045158 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5920e-2e01-4f57-8da2-d2b7d2c9e05f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:22.045465 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5920e-2e01-4f57-8da2-d2b7d2c9e05f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:22.047408 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5920e-2e01-4f57-8da2-d2b7d2c9e05f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:22.064201 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 238/488] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:22.069756 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712cd07-11c9-4e23-a78a-10f1546ba6d5 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:22.070193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5920e-2e01-4f57-8da2-d2b7d2c9e05f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:22.086464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:22.185761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5920e-2e01-4f57-8da2-d2b7d2c9e05f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:22.251558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec372b-5ad9-4ff8-b697-dfc81b5fcdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:22.252256 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec372b-5ad9-4ff8-b697-dfc81b5fcdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:22.252554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec372b-5ad9-4ff8-b697-dfc81b5fcdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:22.252806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec372b-5ad9-4ff8-b697-dfc81b5fcdf3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:22.265694 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712cd07-11c9-4e23-a78a-10f1546ba6d5 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:22.272385 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec372b-5ad9-4ff8-b697-dfc81b5fcdf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:22.277147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-8bec372b-5ad9-4ff8-b697-dfc81b5fcdf3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:22.277147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:22.277147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:22.277147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:22.277147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:22.277147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:22.277147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:22.277147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:22.277147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:22.277147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:22.277147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:22.277147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:22.277147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:22.277147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:22.277147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:22.277147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:22.278047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:22.278047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:22.278047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:22.278047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:22.278047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:22.278047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:22.278047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:22.278047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:22.278047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:22.278047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:22.278047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:22.278047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:22.278047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:22.278047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:22.278047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:22.278047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:22.278902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:22.278902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:22.278902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:22.278902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:22.278902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:22.278902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:22.278902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:22.278902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:22.278902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:22.278902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:22.278902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:22.278902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:22.278902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 239/489] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:22.289347 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec372b-5ad9-4ff8-b697-dfc81b5fcdf3 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:22.323563 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3a18f-0b4e-4626-b42a-1f5c727a2762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:22.324514 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3a18f-0b4e-4626-b42a-1f5c727a2762 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:22.325249 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3a18f-0b4e-4626-b42a-1f5c727a2762 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:22.325529 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3a18f-0b4e-4626-b42a-1f5c727a2762 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:22.334877 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3a18f-0b4e-4626-b42a-1f5c727a2762 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:22.336395 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3a18f-0b4e-4626-b42a-1f5c727a2762 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:22.355366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-b6b3a18f-0b4e-4626-b42a-1f5c727a2762 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:40:22.355366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:22.355366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:22.355366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:22.355366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:22.355366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:22.355366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:22.355366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:22.355366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:22.355366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:22.355366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:22.355366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:22.355366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:22.355366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:22.355366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:22.355366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:22.356700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:22.358024 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:22.358024 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:22.358024 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:22.358024 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:22.358024 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:22.358024 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:22.358024 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:22.358024 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:22.358024 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:40:22.358024 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:22.361864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 251/490] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 3 12:40:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:22.363603 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907daead-7959-4f1d-a6eb-64fb82bab81a None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:22.363944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907daead-7959-4f1d-a6eb-64fb82bab81a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:22.364615 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907daead-7959-4f1d-a6eb-64fb82bab81a None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:22.366038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 252/491] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:22.367269 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907240d-3069-47ca-b94b-e3fdec022fc2 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:22.367556 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907240d-3069-47ca-b94b-e3fdec022fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:22.367828 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907240d-3069-47ca-b94b-e3fdec022fc2 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:22.369124 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 253/492] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:22.370032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907240d-3069-47ca-b94b-e3fdec022fc2 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:22.382467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:22.410359 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec372b-5ad9-4ff8-b697-dfc81b5fcdf3 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:22.436984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:22.446243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd1c83-974e-47dc-b05b-a57fbb97867a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:22.447107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd1c83-974e-47dc-b05b-a57fbb97867a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:22.447385 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd1c83-974e-47dc-b05b-a57fbb97867a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:22.447658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd1c83-974e-47dc-b05b-a57fbb97867a None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:22.448382 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd1c83-974e-47dc-b05b-a57fbb97867a None admin] RBAC: Adding query filter params (name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:22.448691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd1c83-974e-47dc-b05b-a57fbb97867a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:22.450073 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd1c83-974e-47dc-b05b-a57fbb97867a None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:22.458333 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 240/493] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:22.459547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2af38-7f15-48c2-9afd-906eb0ba11fd None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:22.459921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2af38-7f15-48c2-9afd-906eb0ba11fd None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:22.460101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2af38-7f15-48c2-9afd-906eb0ba11fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:22.479727 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-1eb2af38-7f15-48c2-9afd-906eb0ba11fd None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:22.481048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1eb2af38-7f15-48c2-9afd-906eb0ba11fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:22.492390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907240d-3069-47ca-b94b-e3fdec022fc2 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:22.527588 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bf577-77d1-47aa-86b2-14f4a85ca307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:22.528132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bf577-77d1-47aa-86b2-14f4a85ca307 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:22.528418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bf577-77d1-47aa-86b2-14f4a85ca307 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:22.528689 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bf577-77d1-47aa-86b2-14f4a85ca307 None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:22.529475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667bf577-77d1-47aa-86b2-14f4a85ca307 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:22.529791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667bf577-77d1-47aa-86b2-14f4a85ca307 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:22.531745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667bf577-77d1-47aa-86b2-14f4a85ca307 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:22.566506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 241/494] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:22.566506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993644c4-219c-4a62-b462-e8e6ad7ae7cb None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:22.566506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993644c4-219c-4a62-b462-e8e6ad7ae7cb None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:22.566506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993644c4-219c-4a62-b462-e8e6ad7ae7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:22.573426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 254/495] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 3 12:40:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:22.574719 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bf577-77d1-47aa-86b2-14f4a85ca307 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:22.584862 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-993644c4-219c-4a62-b462-e8e6ad7ae7cb None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:22.586422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-993644c4-219c-4a62-b462-e8e6ad7ae7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:22.651852 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:22.662039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 242/496] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:22.662909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993644c4-219c-4a62-b462-e8e6ad7ae7cb None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:22.695130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bf577-77d1-47aa-86b2-14f4a85ca307 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:22.713488 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:22.732461 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26c33e-25f4-4b6b-85a8-d20abdb05eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:22.733110 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26c33e-25f4-4b6b-85a8-d20abdb05eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:22.733389 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26c33e-25f4-4b6b-85a8-d20abdb05eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:22.733658 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26c33e-25f4-4b6b-85a8-d20abdb05eb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:22.743969 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26c33e-25f4-4b6b-85a8-d20abdb05eb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:22.745411 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26c33e-25f4-4b6b-85a8-d20abdb05eb6 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:22.759699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-bd26c33e-25f4-4b6b-85a8-d20abdb05eb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:22.759699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:22.759699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:22.759699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:22.759699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:22.759699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:22.759699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:22.759699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:22.759699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:22.759699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:22.759699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:22.759699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:22.759699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:22.759699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:22.759699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:22.760528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:22.761958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:22.761958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:22.761958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:22.761958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:22.761958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 255/497] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:22.761958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26c33e-25f4-4b6b-85a8-d20abdb05eb6 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:22.775073 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993644c4-219c-4a62-b462-e8e6ad7ae7cb None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:22.808384 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b1e460-44be-4584-bd44-9ae4602b7638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:22.808930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1e460-44be-4584-bd44-9ae4602b7638 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:22.809211 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1e460-44be-4584-bd44-9ae4602b7638 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:22.809488 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b1e460-44be-4584-bd44-9ae4602b7638 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:22.860032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26c33e-25f4-4b6b-85a8-d20abdb05eb6 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:22.871494 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:22.878917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1e460-44be-4584-bd44-9ae4602b7638 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:22.880381 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b1e460-44be-4584-bd44-9ae4602b7638 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:22.895524 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278967a9-da0f-4b30-9f2e-4e8a2e749524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:22.896167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278967a9-da0f-4b30-9f2e-4e8a2e749524 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:22.896445 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278967a9-da0f-4b30-9f2e-4e8a2e749524 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:22.896718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278967a9-da0f-4b30-9f2e-4e8a2e749524 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:22.907368 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278967a9-da0f-4b30-9f2e-4e8a2e749524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:22.912486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-278967a9-da0f-4b30-9f2e-4e8a2e749524 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:22.912486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:22.912486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:22.912486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:22.912486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:22.912486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:22.912486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:22.912486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:22.912486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:22.912486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:22.912486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:22.912486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:22.912486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:22.912486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:22.912486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:22.912486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:22.913567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:22.913567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:22.913567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:22.913567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:22.913567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:22.913567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:22.913567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:22.913567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:22.913567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:22.913567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:22.913567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:22.913567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:22.913567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:22.913567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:22.913567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:22.913567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:22.914468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:22.914468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:22.914468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:22.914468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:22.914468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:22.914468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:22.914468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:22.914468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:22.914468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:22.914468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:22.914468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:22.914468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:22.914468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 256/498] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:22.914468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278967a9-da0f-4b30-9f2e-4e8a2e749524 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:22.941554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:22.984917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 243/499] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:40:22.986188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dc41a-ecb8-444d-803b-e0b1526c8bcb None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:22.986475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dc41a-ecb8-444d-803b-e0b1526c8bcb None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:22.986782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2dc41a-ecb8-444d-803b-e0b1526c8bcb None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:22.988035 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 244/500] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:22.988732 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2dc41a-ecb8-444d-803b-e0b1526c8bcb None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:23.014295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278967a9-da0f-4b30-9f2e-4e8a2e749524 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:23.048139 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e502959-5a7a-41e2-beaa-899d423dc6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:23.048774 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e502959-5a7a-41e2-beaa-899d423dc6de None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:23.049054 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e502959-5a7a-41e2-beaa-899d423dc6de None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:23.049323 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e502959-5a7a-41e2-beaa-899d423dc6de None admin] PATH_INFO: `/v3/roles/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:23.059410 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e502959-5a7a-41e2-beaa-899d423dc6de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:23.064508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-3e502959-5a7a-41e2-beaa-899d423dc6de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:23.064508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:23.064508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:23.064508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:23.064508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:23.064508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:23.064508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:23.064508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:23.064508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:23.064508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:23.064508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:23.064508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:23.064508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:23.064508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:23.064508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:23.064508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:23.065643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:23.065643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:23.065643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:23.065643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:23.065643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:23.065643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:23.065643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:23.065643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:23.065643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:23.065643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:23.065643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:23.065643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:23.065643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:23.065643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:23.065643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:23.065643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:23.066526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:23.066526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:23.066526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:23.066526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:23.066526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:23.066526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:23.066526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:23.066526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:23.066526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:23.066526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:23.066526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:23.066526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:23.066526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 257/501] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:23.067213 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e502959-5a7a-41e2-beaa-899d423dc6de None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:23.088008 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:23.095783 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2dc41a-ecb8-444d-803b-e0b1526c8bcb None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:23.128306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bedb5d-ae6f-4fc4-8687-f78555a5c982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:23.128911 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bedb5d-ae6f-4fc4-8687-f78555a5c982 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:23.129188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bedb5d-ae6f-4fc4-8687-f78555a5c982 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:23.129453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bedb5d-ae6f-4fc4-8687-f78555a5c982 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:23.130138 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bedb5d-ae6f-4fc4-8687-f78555a5c982 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:23.130437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bedb5d-ae6f-4fc4-8687-f78555a5c982 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:23.131833 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bedb5d-ae6f-4fc4-8687-f78555a5c982 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:23.143375 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 245/502] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:23.144141 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bedb5d-ae6f-4fc4-8687-f78555a5c982 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:23.154649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:23.167852 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e502959-5a7a-41e2-beaa-899d423dc6de None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:23.201854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98871e96-0dc7-43a9-96e9-4a12314b668d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:23.202477 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98871e96-0dc7-43a9-96e9-4a12314b668d None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:23.202800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98871e96-0dc7-43a9-96e9-4a12314b668d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:23.203068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98871e96-0dc7-43a9-96e9-4a12314b668d None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:23.204360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98871e96-0dc7-43a9-96e9-4a12314b668d None admin] RBAC: Adding query filter params (name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:23.204671 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98871e96-0dc7-43a9-96e9-4a12314b668d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:23.206037 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98871e96-0dc7-43a9-96e9-4a12314b668d None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:23.213834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 258/503] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:23.214873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf13fb8-cdca-45ae-8191-3c4c875ed83b None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:23.215162 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf13fb8-cdca-45ae-8191-3c4c875ed83b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:23.215455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf13fb8-cdca-45ae-8191-3c4c875ed83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:23.232750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-7cf13fb8-cdca-45ae-8191-3c4c875ed83b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:23.233949 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-7cf13fb8-cdca-45ae-8191-3c4c875ed83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:23.243285 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bedb5d-ae6f-4fc4-8687-f78555a5c982 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:23.276313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a0ee8-8528-413a-a1a0-750e1a73e135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:23.276948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a0ee8-8528-413a-a1a0-750e1a73e135 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:23.277227 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a0ee8-8528-413a-a1a0-750e1a73e135 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:23.277507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a0ee8-8528-413a-a1a0-750e1a73e135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:23.287898 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a0ee8-8528-413a-a1a0-750e1a73e135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:23.289332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a0ee8-8528-413a-a1a0-750e1a73e135 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:23.298800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 259/504] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:23.299616 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf13fb8-cdca-45ae-8191-3c4c875ed83b None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:23.303191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-338a0ee8-8528-413a-a1a0-750e1a73e135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:23.303191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:23.303191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:23.303191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:23.303191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:23.303191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:23.303191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:23.303191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:23.303191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:23.303191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:23.303191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:23.303191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:23.303191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:23.303191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:23.303191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:23.304179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:23.305045 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:23.305045 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:23.305045 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:23.305045 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:23.305045 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 246/505] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:23.305045 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a0ee8-8528-413a-a1a0-750e1a73e135 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:23.317541 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:23.392698 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:23.406044 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a0ee8-8528-413a-a1a0-750e1a73e135 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:23.412152 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf13fb8-cdca-45ae-8191-3c4c875ed83b None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:23.439123 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057c9ad-dfc8-47cc-baa3-76d265788958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:23.439751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057c9ad-dfc8-47cc-baa3-76d265788958 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:23.440025 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057c9ad-dfc8-47cc-baa3-76d265788958 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:23.440292 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057c9ad-dfc8-47cc-baa3-76d265788958 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:23.446003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582861a-d9b6-4cbf-840d-131a99c606da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:23.446656 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582861a-d9b6-4cbf-840d-131a99c606da None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:23.446938 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582861a-d9b6-4cbf-840d-131a99c606da None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:23.447215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582861a-d9b6-4cbf-840d-131a99c606da None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:23.447616 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057c9ad-dfc8-47cc-baa3-76d265788958 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:23.447929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582861a-d9b6-4cbf-840d-131a99c606da None admin] RBAC: Adding query filter params (name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:23.448237 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582861a-d9b6-4cbf-840d-131a99c606da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:23.449075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057c9ad-dfc8-47cc-baa3-76d265788958 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:23.449587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582861a-d9b6-4cbf-840d-131a99c606da None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:23.458012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 260/506] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:23.458836 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582861a-d9b6-4cbf-840d-131a99c606da None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:23.462174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-e057c9ad-dfc8-47cc-baa3-76d265788958 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 03 12:40:23.462174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:23.462174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:23.462174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:23.462174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:23.462174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:23.462174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:23.462174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:23.462174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:23.462174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:23.462174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:23.462174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:23.462174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:23.462174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:23.462174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:23.462174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:23.463286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:23.464240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:23.464240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:23.464240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:23.464240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:23.464240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:23.464240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:23.464240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:23.464240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:23.464240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 03 12:40:23.464240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:23.464240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 247/507] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:23.464240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057c9ad-dfc8-47cc-baa3-76d265788958 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:23.538331 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:23.565625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582861a-d9b6-4cbf-840d-131a99c606da None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:23.579723 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057c9ad-dfc8-47cc-baa3-76d265788958 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:23.597894 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458a60a-70bb-4f11-8ea0-69a4ddf8f8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:23.598548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458a60a-70bb-4f11-8ea0-69a4ddf8f8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:23.598827 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458a60a-70bb-4f11-8ea0-69a4ddf8f8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:23.599097 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f458a60a-70bb-4f11-8ea0-69a4ddf8f8c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:23.610059 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458a60a-70bb-4f11-8ea0-69a4ddf8f8c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:23.611540 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f458a60a-70bb-4f11-8ea0-69a4ddf8f8c8 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:23.612372 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b932192-8fdd-4591-a850-02a19586f455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:23.612972 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b932192-8fdd-4591-a850-02a19586f455 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:23.613256 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b932192-8fdd-4591-a850-02a19586f455 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:23.613353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:23.613532 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b932192-8fdd-4591-a850-02a19586f455 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:23.614228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b932192-8fdd-4591-a850-02a19586f455 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:23.614671 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b932192-8fdd-4591-a850-02a19586f455 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:23.616042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b932192-8fdd-4591-a850-02a19586f455 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:23.627351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-f458a60a-70bb-4f11-8ea0-69a4ddf8f8c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:23.627351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:23.627351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:23.627351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:23.627351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:23.627351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:23.627351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:23.627351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:23.627351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:23.627351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:23.627351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:23.627351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:23.627351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:23.627351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:23.627351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:23.628414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:23.629724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:23.629724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:23.629724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:23.629724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:23.629724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 261/509] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:23.629724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458a60a-70bb-4f11-8ea0-69a4ddf8f8c8 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:23.630539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 248/509] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:23.631519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b932192-8fdd-4591-a850-02a19586f455 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:23.726299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f458a60a-70bb-4f11-8ea0-69a4ddf8f8c8 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:23.728919 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b932192-8fdd-4591-a850-02a19586f455 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:23.755552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:23.760149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c9825-4a0d-4dd3-b1ec-ff4953ecb0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:23.760782 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c9825-4a0d-4dd3-b1ec-ff4953ecb0cc None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:23.761060 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c9825-4a0d-4dd3-b1ec-ff4953ecb0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:23.761330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c9825-4a0d-4dd3-b1ec-ff4953ecb0cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:23.763554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0e44c-958c-4146-ac3a-5d5d43c4bb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:23.764178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0e44c-958c-4146-ac3a-5d5d43c4bb63 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:23.764455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0e44c-958c-4146-ac3a-5d5d43c4bb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:23.764723 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0e44c-958c-4146-ac3a-5d5d43c4bb63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:23.772200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c9825-4a0d-4dd3-b1ec-ff4953ecb0cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:23.773654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c9825-4a0d-4dd3-b1ec-ff4953ecb0cc None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:23.775118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0e44c-958c-4146-ac3a-5d5d43c4bb63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:23.776539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a0e44c-958c-4146-ac3a-5d5d43c4bb63 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:23.788267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-021c9825-4a0d-4dd3-b1ec-ff4953ecb0cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:23.788267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:23.788267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:23.788267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:23.788267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:23.788267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:23.788267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:23.788267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:23.788267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:23.788267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:23.788267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:23.788267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:23.788267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:23.788267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:23.788267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:23.789200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:23.790058 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:23.790058 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:23.790058 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:23.790058 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:23.790058 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 262/510] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:23.790567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c9825-4a0d-4dd3-b1ec-ff4953ecb0cc None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:23.791476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-36a0e44c-958c-4146-ac3a-5d5d43c4bb63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:23.791476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:23.791476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:23.791476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:23.791476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:23.791476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:23.791476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:23.791476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:23.791476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:23.791476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:23.791476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:23.791476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:23.791476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:23.791476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:23.791476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:23.792280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:23.793134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:23.793134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:23.793134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:23.793134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:23.793134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 249/511] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:23.793392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0e44c-958c-4146-ac3a-5d5d43c4bb63 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:23.837165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:23.892413 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c9825-4a0d-4dd3-b1ec-ff4953ecb0cc None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:23.897146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0e44c-958c-4146-ac3a-5d5d43c4bb63 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:23.925162 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d25c7-c311-4b46-8f4a-93db96f6cbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:23.925772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d25c7-c311-4b46-8f4a-93db96f6cbcb None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:23.926078 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d25c7-c311-4b46-8f4a-93db96f6cbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:23.926351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954d25c7-c311-4b46-8f4a-93db96f6cbcb None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:23.927201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d25c7-c311-4b46-8f4a-93db96f6cbcb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:23.927508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d25c7-c311-4b46-8f4a-93db96f6cbcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:23.928930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954d25c7-c311-4b46-8f4a-93db96f6cbcb None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:23.929298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca20c3d-2bc4-49d6-9040-ccce20ba13c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:23.929922 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca20c3d-2bc4-49d6-9040-ccce20ba13c8 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:23.930202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca20c3d-2bc4-49d6-9040-ccce20ba13c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:23.930466 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca20c3d-2bc4-49d6-9040-ccce20ba13c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:23.937153 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca20c3d-2bc4-49d6-9040-ccce20ba13c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:23.939093 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca20c3d-2bc4-49d6-9040-ccce20ba13c8 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:23.946234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 263/512] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 3 12:40:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:23.946976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d25c7-c311-4b46-8f4a-93db96f6cbcb None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:23.950647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-1ca20c3d-2bc4-49d6-9040-ccce20ba13c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:23.950647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:23.950647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:23.950647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:23.950647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:23.950647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:23.950647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:23.950647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:23.950647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:23.950647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:23.950647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:23.950647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:23.950647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:23.950647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:23.950647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:23.951506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:23.952364 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:23.952364 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:23.952364 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:23.952364 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:23.952364 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:23.952364 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:23.952364 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:23.952364 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:23.954790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 250/513] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:23.955505 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca20c3d-2bc4-49d6-9040-ccce20ba13c8 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:23.994982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:24.045446 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954d25c7-c311-4b46-8f4a-93db96f6cbcb None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:24.047711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:24.054001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca20c3d-2bc4-49d6-9040-ccce20ba13c8 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:24.080128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ac1bc-3aad-4d96-bab3-4703133e66ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:24.080747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ac1bc-3aad-4d96-bab3-4703133e66ec None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:24.081029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ac1bc-3aad-4d96-bab3-4703133e66ec None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:24.081308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ac1bc-3aad-4d96-bab3-4703133e66ec None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:24.082003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ac1bc-3aad-4d96-bab3-4703133e66ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:24.082311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ac1bc-3aad-4d96-bab3-4703133e66ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:24.083714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ac1bc-3aad-4d96-bab3-4703133e66ec None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:24.088119 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789e247-e9bf-46fa-ae09-bd3080cf9627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:24.088675 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789e247-e9bf-46fa-ae09-bd3080cf9627 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:24.088953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789e247-e9bf-46fa-ae09-bd3080cf9627 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:24.089225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789e247-e9bf-46fa-ae09-bd3080cf9627 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:24.089916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789e247-e9bf-46fa-ae09-bd3080cf9627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:24.090222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789e247-e9bf-46fa-ae09-bd3080cf9627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:24.091599 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789e247-e9bf-46fa-ae09-bd3080cf9627 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:24.094530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 264/514] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:24.095315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ac1bc-3aad-4d96-bab3-4703133e66ec None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:24.104101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 251/515] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:24.104843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789e247-e9bf-46fa-ae09-bd3080cf9627 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:24.195282 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ac1bc-3aad-4d96-bab3-4703133e66ec None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:24.200921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:24.204289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789e247-e9bf-46fa-ae09-bd3080cf9627 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:24.228219 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d770f2-c612-4c08-bb9d-da28d2642c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:24.228833 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d770f2-c612-4c08-bb9d-da28d2642c94 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:24.229118 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d770f2-c612-4c08-bb9d-da28d2642c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:24.229399 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d770f2-c612-4c08-bb9d-da28d2642c94 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:24.230095 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d770f2-c612-4c08-bb9d-da28d2642c94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:24.230403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d770f2-c612-4c08-bb9d-da28d2642c94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:24.231895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d770f2-c612-4c08-bb9d-da28d2642c94 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:24.240601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3b9d7-55ad-4238-9a5e-8c373ac1242c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:24.241205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3b9d7-55ad-4238-9a5e-8c373ac1242c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:24.241486 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3b9d7-55ad-4238-9a5e-8c373ac1242c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:24.241760 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3b9d7-55ad-4238-9a5e-8c373ac1242c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:24.247772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 265/516] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:24.247772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d770f2-c612-4c08-bb9d-da28d2642c94 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:24.271684 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:24.308533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3b9d7-55ad-4238-9a5e-8c373ac1242c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:24.310257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3b9d7-55ad-4238-9a5e-8c373ac1242c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:24.345009 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d770f2-c612-4c08-bb9d-da28d2642c94 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:24.378601 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45375bf1-9a96-4472-bf39-f452dd2d5a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:24.379210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45375bf1-9a96-4472-bf39-f452dd2d5a72 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:24.379493 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45375bf1-9a96-4472-bf39-f452dd2d5a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:24.379765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45375bf1-9a96-4472-bf39-f452dd2d5a72 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:24.380513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45375bf1-9a96-4472-bf39-f452dd2d5a72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:24.380819 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45375bf1-9a96-4472-bf39-f452dd2d5a72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:24.382286 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45375bf1-9a96-4472-bf39-f452dd2d5a72 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:24.399571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 266/517] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:24.400260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45375bf1-9a96-4472-bf39-f452dd2d5a72 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:24.408082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:24.412098 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 252/518] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:40:24.413094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3b9d7-55ad-4238-9a5e-8c373ac1242c None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:24.483888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:24.501062 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45375bf1-9a96-4472-bf39-f452dd2d5a72 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:24.509936 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3b9d7-55ad-4238-9a5e-8c373ac1242c None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:24.534174 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce088244-2f83-4dce-9aec-181029d1f226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:24.534843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce088244-2f83-4dce-9aec-181029d1f226 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:24.535123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce088244-2f83-4dce-9aec-181029d1f226 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:24.535403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce088244-2f83-4dce-9aec-181029d1f226 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:24.542330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce088244-2f83-4dce-9aec-181029d1f226 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:24.543830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce088244-2f83-4dce-9aec-181029d1f226 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:24.545617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be33d6-2154-44d9-9c5e-a735070213b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:24.546216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be33d6-2154-44d9-9c5e-a735070213b9 None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:24.546512 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be33d6-2154-44d9-9c5e-a735070213b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:24.547314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be33d6-2154-44d9-9c5e-a735070213b9 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:24.548320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be33d6-2154-44d9-9c5e-a735070213b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:24.550016 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be33d6-2154-44d9-9c5e-a735070213b9 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:24.555805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-ce088244-2f83-4dce-9aec-181029d1f226 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 12:40:24.555805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:24.555805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:24.555805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:24.555805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:24.555805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:24.555805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:24.555805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:24.555805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:24.555805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:24.555805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:24.555805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:24.555805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:24.555805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:24.555805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:24.555805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:24.556886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:24.557816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:24.557816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:24.557816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:24.557816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:24.557816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:24.557816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:24.557816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:24.557816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:24.557816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 12:40:24.557816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:24.557816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 267/519] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:24.558381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce088244-2f83-4dce-9aec-181029d1f226 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:24.574824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 253/520] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:40:24 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:40:24.574824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be33d6-2154-44d9-9c5e-a735070213b9 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:24.653311 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:24.669357 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce088244-2f83-4dce-9aec-181029d1f226 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:24.704246 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be33d6-2154-44d9-9c5e-a735070213b9 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:24.706113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e3670-bd6b-4619-9b0e-771e8374fad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:24.706816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e3670-bd6b-4619-9b0e-771e8374fad9 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:24.707094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e3670-bd6b-4619-9b0e-771e8374fad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:24.707368 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e3670-bd6b-4619-9b0e-771e8374fad9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:24.715697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e3670-bd6b-4619-9b0e-771e8374fad9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:24.717175 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e3670-bd6b-4619-9b0e-771e8374fad9 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:24.730334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-131e3670-bd6b-4619-9b0e-771e8374fad9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 12:40:24.730334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:24.730334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:24.730334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:24.730334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:24.730334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:24.730334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:24.730334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:24.730334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:24.730334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:24.730334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:24.730334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:24.730334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:24.730334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:24.730334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:24.730334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:24.732381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:24.733520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:24.733520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:24.733520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:24.733520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:24.733520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:24.733520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:24.733520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:24.733520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:24.733520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 12:40:24.733520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:24.733520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 268/521] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:24.733520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e3670-bd6b-4619-9b0e-771e8374fad9 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:24.736429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:24.742376 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab02a1-a9e8-42a8-bfdf-0405664f7147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:24.743086 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab02a1-a9e8-42a8-bfdf-0405664f7147 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:24.743364 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab02a1-a9e8-42a8-bfdf-0405664f7147 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:24.743685 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab02a1-a9e8-42a8-bfdf-0405664f7147 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:24.744477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab02a1-a9e8-42a8-bfdf-0405664f7147 None admin] RBAC: Adding query filter params (role.id=ac03c5aaca774e3196210f3bdb442418, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=7b634f567dca42dc95df175a4f36c1d1) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:24.745365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab02a1-a9e8-42a8-bfdf-0405664f7147 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:24.746445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ab02a1-a9e8-42a8-bfdf-0405664f7147 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:24.752915 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 254/522] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:40:24 2022] GET /identity/v3/role_assignments?user.id=7b634f567dca42dc95df175a4f36c1d1&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=ac03c5aaca774e3196210f3bdb442418 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:24.753924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab02a1-a9e8-42a8-bfdf-0405664f7147 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:24.857575 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e3670-bd6b-4619-9b0e-771e8374fad9 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:24.901630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab02a1-a9e8-42a8-bfdf-0405664f7147 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:24.916448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc1555-8a76-47cc-b2de-1426dd1a92bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:24.917030 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc1555-8a76-47cc-b2de-1426dd1a92bc None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:24.917463 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc1555-8a76-47cc-b2de-1426dd1a92bc None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:24.917746 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc1555-8a76-47cc-b2de-1426dd1a92bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:24.926839 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:24.932369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc1555-8a76-47cc-b2de-1426dd1a92bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:24.935114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc1555-8a76-47cc-b2de-1426dd1a92bc None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:24.941842 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5dea7-fc90-4c15-9048-62606784d722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:24.942468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5dea7-fc90-4c15-9048-62606784d722 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:24.942798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5dea7-fc90-4c15-9048-62606784d722 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:24.943122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5dea7-fc90-4c15-9048-62606784d722 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:24.943940 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5dea7-fc90-4c15-9048-62606784d722 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:24.944254 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5dea7-fc90-4c15-9048-62606784d722 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:24.945759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5dea7-fc90-4c15-9048-62606784d722 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:24.951228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-d1dc1555-8a76-47cc-b2de-1426dd1a92bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:24.951228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:24.951228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:24.951228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:24.951228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:24.951228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:24.951228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:24.951228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:24.951228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:24.951228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:24.951228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:24.951228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:24.951228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:24.951228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:24.951228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:24.952055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:24.953442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:24.953442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:24.953442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:24.953442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:24.953442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 269/523] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:24.953442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc1555-8a76-47cc-b2de-1426dd1a92bc None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:24.994599 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 255/524] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 3 12:40:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:24.995966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5dea7-fc90-4c15-9048-62606784d722 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:25.018715 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:25.074378 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc1555-8a76-47cc-b2de-1426dd1a92bc None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:25.106034 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5dea7-fc90-4c15-9048-62606784d722 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:25.125163 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c0143-9ba0-45b6-bf14-ad7378b707be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:25.126458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c0143-9ba0-45b6-bf14-ad7378b707be None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:25.126998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c0143-9ba0-45b6-bf14-ad7378b707be None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:25.127692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c0143-9ba0-45b6-bf14-ad7378b707be None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:25.129119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c0143-9ba0-45b6-bf14-ad7378b707be None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:25.129623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c0143-9ba0-45b6-bf14-ad7378b707be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:25.131816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c0143-9ba0-45b6-bf14-ad7378b707be None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:25.146613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3bd24-9be8-469f-9fd0-3a1c4bd19056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:25.147241 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3bd24-9be8-469f-9fd0-3a1c4bd19056 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:25.147524 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3bd24-9be8-469f-9fd0-3a1c4bd19056 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:25.147798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3bd24-9be8-469f-9fd0-3a1c4bd19056 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:25.148566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3bd24-9be8-469f-9fd0-3a1c4bd19056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:25.148873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3bd24-9be8-469f-9fd0-3a1c4bd19056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:25.150326 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3bd24-9be8-469f-9fd0-3a1c4bd19056 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:25.162255 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 270/525] 10.222.0.65 () {56 vars in 1139 bytes} [Wed Aug 3 12:40:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:25.162255 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c0143-9ba0-45b6-bf14-ad7378b707be None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:25.173749 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 256/526] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:25.174639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3bd24-9be8-469f-9fd0-3a1c4bd19056 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:25.218343 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:25.283472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3bd24-9be8-469f-9fd0-3a1c4bd19056 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:25.297313 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c0143-9ba0-45b6-bf14-ad7378b707be None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:25.318253 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:25.321504 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41723a0a-c050-4f97-979f-d0bc220737d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:25.322206 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41723a0a-c050-4f97-979f-d0bc220737d9 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:25.322605 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41723a0a-c050-4f97-979f-d0bc220737d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:25.323028 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41723a0a-c050-4f97-979f-d0bc220737d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:25.346519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c0ae0-2613-4645-b219-c6c30dbf4d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:25.347475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c0ae0-2613-4645-b219-c6c30dbf4d31 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:25.348403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c0ae0-2613-4645-b219-c6c30dbf4d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:25.348822 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c0ae0-2613-4645-b219-c6c30dbf4d31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:25.397456 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41723a0a-c050-4f97-979f-d0bc220737d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:25.399149 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41723a0a-c050-4f97-979f-d0bc220737d9 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:25.441413 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:25.445810 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c0ae0-2613-4645-b219-c6c30dbf4d31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:25.447758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c0ae0-2613-4645-b219-c6c30dbf4d31 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:25.513127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 257/527] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:40:25.514250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41723a0a-c050-4f97-979f-d0bc220737d9 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:25.576868 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 271/528] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:40:25.578775 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c0ae0-2613-4645-b219-c6c30dbf4d31 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:25.597669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:25.646686 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41723a0a-c050-4f97-979f-d0bc220737d9 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:25.684141 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25cc2b-1c19-4eab-97b7-4ec1129e533b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:25.685184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25cc2b-1c19-4eab-97b7-4ec1129e533b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:25.685563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25cc2b-1c19-4eab-97b7-4ec1129e533b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:25.685939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25cc2b-1c19-4eab-97b7-4ec1129e533b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:25.693601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25cc2b-1c19-4eab-97b7-4ec1129e533b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:25.695186 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25cc2b-1c19-4eab-97b7-4ec1129e533b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:25.696997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:25.701545 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c0ae0-2613-4645-b219-c6c30dbf4d31 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:25.709704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-de25cc2b-1c19-4eab-97b7-4ec1129e533b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:25.709704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:25.709704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:25.709704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:25.709704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:25.709704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:25.709704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:25.709704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:25.709704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:25.709704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:25.709704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:25.709704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:25.709704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:25.709704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:25.709704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:25.710781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:25.711902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:25.711902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:25.711902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:25.711902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:25.711902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:25.711902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:25.711902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:25.711902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:25.718440 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 258/529] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:25.719437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25cc2b-1c19-4eab-97b7-4ec1129e533b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:25.748334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb04ce-95ef-4ff8-88e9-dddcb276c5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:25.748985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb04ce-95ef-4ff8-88e9-dddcb276c5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:25.749271 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb04ce-95ef-4ff8-88e9-dddcb276c5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:25.749545 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb04ce-95ef-4ff8-88e9-dddcb276c5e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:25.760783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb04ce-95ef-4ff8-88e9-dddcb276c5e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:25.762236 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb04ce-95ef-4ff8-88e9-dddcb276c5e2 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:25.778157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-beeb04ce-95ef-4ff8-88e9-dddcb276c5e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:25.778157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:25.778157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:25.778157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:25.778157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:25.778157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:25.778157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:25.778157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:25.778157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:25.778157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:25.778157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:25.778157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:25.778157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:25.778157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:25.778157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:25.779138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:25.780004 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:25.780004 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:25.780004 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:25.780004 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:25.780004 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 272/530] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:25.780288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb04ce-95ef-4ff8-88e9-dddcb276c5e2 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:25.838348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25cc2b-1c19-4eab-97b7-4ec1129e533b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:25.860256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:25.878642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedebbe-f42a-46e3-a737-a205a5e565da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:25.879652 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedebbe-f42a-46e3-a737-a205a5e565da None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:25.880784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedebbe-f42a-46e3-a737-a205a5e565da None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:25.881252 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfedebbe-f42a-46e3-a737-a205a5e565da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:25.892921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfedebbe-f42a-46e3-a737-a205a5e565da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:25.894870 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfedebbe-f42a-46e3-a737-a205a5e565da None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:25.897890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb04ce-95ef-4ff8-88e9-dddcb276c5e2 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:25.911619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-dfedebbe-f42a-46e3-a737-a205a5e565da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:25.911619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:25.911619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:25.911619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:25.911619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:25.911619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:25.911619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:25.911619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:25.911619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:25.911619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:25.911619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:25.911619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:25.911619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:25.911619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:25.911619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:25.912526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:25.913374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:25.913374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:25.913374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:25.913374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:25.915600 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 259/531] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:25.916641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedebbe-f42a-46e3-a737-a205a5e565da None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:25.940592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145bdab-edda-4483-8279-4138aab5b61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:25.941218 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145bdab-edda-4483-8279-4138aab5b61e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:25.941499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145bdab-edda-4483-8279-4138aab5b61e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:25.941774 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145bdab-edda-4483-8279-4138aab5b61e None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:25.942902 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145bdab-edda-4483-8279-4138aab5b61e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:25.942902 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145bdab-edda-4483-8279-4138aab5b61e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:25.944470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145bdab-edda-4483-8279-4138aab5b61e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:25.962099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 273/532] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:25.962099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145bdab-edda-4483-8279-4138aab5b61e None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:25.975295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:26.037113 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfedebbe-f42a-46e3-a737-a205a5e565da None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:26.081002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31c54b-6156-4e6c-a4e4-a19e77acf35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:26.081449 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31c54b-6156-4e6c-a4e4-a19e77acf35b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:26.081732 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31c54b-6156-4e6c-a4e4-a19e77acf35b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:26.082005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31c54b-6156-4e6c-a4e4-a19e77acf35b None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:26.082742 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec31c54b-6156-4e6c-a4e4-a19e77acf35b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:26.083046 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec31c54b-6156-4e6c-a4e4-a19e77acf35b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:26.084414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec31c54b-6156-4e6c-a4e4-a19e77acf35b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:26.100430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 260/533] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:26.100430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31c54b-6156-4e6c-a4e4-a19e77acf35b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:26.104352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145bdab-edda-4483-8279-4138aab5b61e None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:26.131481 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:26.141599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb203085-bd5b-4264-ad0f-7356356d1111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:26.142296 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb203085-bd5b-4264-ad0f-7356356d1111 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:26.142505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb203085-bd5b-4264-ad0f-7356356d1111 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:26.142804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb203085-bd5b-4264-ad0f-7356356d1111 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:26.143494 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb203085-bd5b-4264-ad0f-7356356d1111 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:26.143806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb203085-bd5b-4264-ad0f-7356356d1111 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:26.145216 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb203085-bd5b-4264-ad0f-7356356d1111 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:26.162501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 274/534] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:26.163540 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb203085-bd5b-4264-ad0f-7356356d1111 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:26.231338 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31c54b-6156-4e6c-a4e4-a19e77acf35b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:26.246930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:26.267188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb203085-bd5b-4264-ad0f-7356356d1111 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:26.279604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e8da7-49b5-40fd-aee5-b4f648663104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:26.280691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e8da7-49b5-40fd-aee5-b4f648663104 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:26.281701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e8da7-49b5-40fd-aee5-b4f648663104 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:26.282576 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e8da7-49b5-40fd-aee5-b4f648663104 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:26.283848 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e8da7-49b5-40fd-aee5-b4f648663104 None admin] RBAC: Adding query filter params (role.id=ac03c5aaca774e3196210f3bdb442418, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=e6066e48aa644a47a889e09cfe204858) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:26.284493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e8da7-49b5-40fd-aee5-b4f648663104 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:26.286448 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e8da7-49b5-40fd-aee5-b4f648663104 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:26.302351 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 261/535] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:40:26 2022] GET /identity/v3/role_assignments?user.id=e6066e48aa644a47a889e09cfe204858&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=ac03c5aaca774e3196210f3bdb442418 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:26.302351 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e8da7-49b5-40fd-aee5-b4f648663104 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:26.320057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0d972-804b-482d-9508-e8be9998fbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:26.320783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0d972-804b-482d-9508-e8be9998fbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:26.321064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0d972-804b-482d-9508-e8be9998fbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:26.321338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0d972-804b-482d-9508-e8be9998fbe1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:26.334887 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0d972-804b-482d-9508-e8be9998fbe1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:26.338087 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0d972-804b-482d-9508-e8be9998fbe1 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:26.364553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-3cc0d972-804b-482d-9508-e8be9998fbe1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:26.364553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:26.364553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:26.364553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:26.364553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:26.364553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:26.364553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:26.364553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:26.364553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:26.364553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:26.364553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:26.364553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:26.364553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:26.364553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:26.364553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:26.365484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:26.366351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:26.366351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:26.366351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:26.366351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:26.366351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:26.366351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:26.366351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:26.366351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:26.366351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 275/536] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:26.374295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0d972-804b-482d-9508-e8be9998fbe1 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:26.476716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:26.511862 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e8da7-49b5-40fd-aee5-b4f648663104 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:26.556462 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0d972-804b-482d-9508-e8be9998fbe1 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:26.563752 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e95aa-ce5b-4608-9c8b-830d82cef337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:26.564413 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e95aa-ce5b-4608-9c8b-830d82cef337 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:26.564691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e95aa-ce5b-4608-9c8b-830d82cef337 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:26.564960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e95aa-ce5b-4608-9c8b-830d82cef337 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:26.579357 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e95aa-ce5b-4608-9c8b-830d82cef337 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:26.579720 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1e95aa-ce5b-4608-9c8b-830d82cef337 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:26.603733 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e5de6-91c3-413a-b572-95544a989751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:26.604720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e5de6-91c3-413a-b572-95544a989751 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:26.605192 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e5de6-91c3-413a-b572-95544a989751 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:26.605665 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9e5de6-91c3-413a-b572-95544a989751 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:26.606627 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e5de6-91c3-413a-b572-95544a989751 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:26.607642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-aa1e95aa-ce5b-4608-9c8b-830d82cef337 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:26.607642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:26.607642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:26.607642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:26.607642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:26.607642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:26.607642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:26.607642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:26.607642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:26.607642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:26.607642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:26.607642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:26.607642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:26.607642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:26.607642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:26.608457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:26.609431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:26.609431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:26.609431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:26.609431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:26.609431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:26.609431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:26.609431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:26.609431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:26.610666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e5de6-91c3-413a-b572-95544a989751 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:26.612297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9e5de6-91c3-413a-b572-95544a989751 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:26.613778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 262/537] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:26.619528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e95aa-ce5b-4608-9c8b-830d82cef337 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:26.638277 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 276/538] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:26.645238 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e5de6-91c3-413a-b572-95544a989751 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:26.688156 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:26.765094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e95aa-ce5b-4608-9c8b-830d82cef337 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:26.783696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9e5de6-91c3-413a-b572-95544a989751 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:26.792310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:26.807424 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fb452-14d5-4b94-b8e6-a12e746e2d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:26.808210 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fb452-14d5-4b94-b8e6-a12e746e2d74 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:26.808590 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fb452-14d5-4b94-b8e6-a12e746e2d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:26.808982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fb452-14d5-4b94-b8e6-a12e746e2d74 None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:26.810318 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fb452-14d5-4b94-b8e6-a12e746e2d74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:26.810628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fb452-14d5-4b94-b8e6-a12e746e2d74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:26.812159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fb452-14d5-4b94-b8e6-a12e746e2d74 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:26.825725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6d47c6-04c2-4407-8e29-2d958e0cef21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:26.826583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d47c6-04c2-4407-8e29-2d958e0cef21 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:26.827002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d47c6-04c2-4407-8e29-2d958e0cef21 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:26.827414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d47c6-04c2-4407-8e29-2d958e0cef21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:26.829336 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d47c6-04c2-4407-8e29-2d958e0cef21 None admin] RBAC: Adding query filter params (role.id=ac03c5aaca774e3196210f3bdb442418, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=52ff458fd7f74b99a745edbcf13a114b) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:26.830034 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d47c6-04c2-4407-8e29-2d958e0cef21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:26.831635 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6d47c6-04c2-4407-8e29-2d958e0cef21 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:26.831744 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 263/539] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:26.833042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199e519-a579-4ba0-847e-1d46202bee7f None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:26.833331 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199e519-a579-4ba0-847e-1d46202bee7f None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:26.833607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199e519-a579-4ba0-847e-1d46202bee7f None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:26.838805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 264/540] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:26.838805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8c5a2-1577-45e1-9582-21498ee50ce1 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:26.838805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8c5a2-1577-45e1-9582-21498ee50ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:26.838805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8c5a2-1577-45e1-9582-21498ee50ce1 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:26.838805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 265/541] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:26.842389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8c5a2-1577-45e1-9582-21498ee50ce1 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:26.842598 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 277/542] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:40:26 2022] GET /identity/v3/role_assignments?user.id=52ff458fd7f74b99a745edbcf13a114b&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=ac03c5aaca774e3196210f3bdb442418 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:26.844666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd3de8-37f6-47c5-bdb9-29fca66285ee None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:26.845052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd3de8-37f6-47c5-bdb9-29fca66285ee None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:26.845500 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd3de8-37f6-47c5-bdb9-29fca66285ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:26.866987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-5cdd3de8-37f6-47c5-bdb9-29fca66285ee None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:26.868228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5cdd3de8-37f6-47c5-bdb9-29fca66285ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:26.955789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 278/543] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:26.957261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76064ba-d165-47ec-9d32-4d6cc98b5618 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:26.957553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76064ba-d165-47ec-9d32-4d6cc98b5618 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:26.957831 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76064ba-d165-47ec-9d32-4d6cc98b5618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:26.977076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-d76064ba-d165-47ec-9d32-4d6cc98b5618 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:26.978318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d76064ba-d165-47ec-9d32-4d6cc98b5618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:26.983428 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:26.993076 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8c5a2-1577-45e1-9582-21498ee50ce1 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:27.040069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b934e-0d42-4246-b443-cf3dd0c853d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:27.040626 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b934e-0d42-4246-b443-cf3dd0c853d5 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:27.041153 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b934e-0d42-4246-b443-cf3dd0c853d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:27.041438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b934e-0d42-4246-b443-cf3dd0c853d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:27.042691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b934e-0d42-4246-b443-cf3dd0c853d5 None admin] RBAC: Adding query filter params (role.id=ac03c5aaca774e3196210f3bdb442418, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=473a5e7e980b4473acec85905fa7078c) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:27.043009 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b934e-0d42-4246-b443-cf3dd0c853d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:27.044470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b934e-0d42-4246-b443-cf3dd0c853d5 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:27.054573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 266/544] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:40:26 2022] GET /identity/v3/role_assignments?user.id=473a5e7e980b4473acec85905fa7078c&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=ac03c5aaca774e3196210f3bdb442418 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:27.055419 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b934e-0d42-4246-b443-cf3dd0c853d5 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:27.072374 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 279/545] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:27.107279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76064ba-d165-47ec-9d32-4d6cc98b5618 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:27.165207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:27.244868 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b934e-0d42-4246-b443-cf3dd0c853d5 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:27.287853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76064ba-d165-47ec-9d32-4d6cc98b5618 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:27.289818 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769af04d-ef74-4a19-bb06-5dadb3d0cdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:27.290750 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769af04d-ef74-4a19-bb06-5dadb3d0cdcd None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:27.291181 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769af04d-ef74-4a19-bb06-5dadb3d0cdcd None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:27.291657 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769af04d-ef74-4a19-bb06-5dadb3d0cdcd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:27.304606 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769af04d-ef74-4a19-bb06-5dadb3d0cdcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:27.310510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-769af04d-ef74-4a19-bb06-5dadb3d0cdcd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:27.310510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:27.310510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:27.310510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:27.310510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:27.310510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:27.310510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:27.310510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:27.310510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:27.310510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:27.310510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:27.310510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:27.310510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:27.310510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:27.310510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:27.310510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:27.311617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:27.311617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:27.311617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:27.311617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:27.311617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:27.311617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:27.311617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:27.311617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:27.311617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:27.311617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:27.311617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:27.311617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:27.311617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:27.311617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:27.311617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:27.311617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:27.313154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:27.313154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:27.313154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:27.313154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:27.313154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:27.313154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:27.313154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:27.313154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:27.313154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:27.313154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:27.313154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:27.313154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:27.316171 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 267/546] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:27.322048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769af04d-ef74-4a19-bb06-5dadb3d0cdcd None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:27.359762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:27.369309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae098f0-0858-4dfe-8b33-cac9bcc523ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:27.369967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae098f0-0858-4dfe-8b33-cac9bcc523ec None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:27.370536 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae098f0-0858-4dfe-8b33-cac9bcc523ec None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:27.370823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae098f0-0858-4dfe-8b33-cac9bcc523ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:27.388402 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae098f0-0858-4dfe-8b33-cac9bcc523ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:27.395623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-5ae098f0-0858-4dfe-8b33-cac9bcc523ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:27.395623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:27.395623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:27.395623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:27.395623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:27.395623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:27.395623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:27.395623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:27.395623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:27.395623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:27.395623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:27.395623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:27.395623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:27.395623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:27.395623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:27.395623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:27.396678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:27.396678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:27.396678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:27.396678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:27.396678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:27.396678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:27.396678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:27.396678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:27.396678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:27.396678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:27.396678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:27.396678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:27.396678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:27.396678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:27.396678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:27.396678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:27.397649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:27.397649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:27.397649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:27.397649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:27.397649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:27.397649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:27.397649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:27.397649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:27.397649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:27.397649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:27.397649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:27.397649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:27.398285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 280/547] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:27.409171 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae098f0-0858-4dfe-8b33-cac9bcc523ec None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:27.530398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769af04d-ef74-4a19-bb06-5dadb3d0cdcd None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:27.581901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdd083-44a9-480b-9c2e-409f1594ee5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:27.582616 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdd083-44a9-480b-9c2e-409f1594ee5c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:27.582904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdd083-44a9-480b-9c2e-409f1594ee5c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:27.583180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdd083-44a9-480b-9c2e-409f1594ee5c None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:27.583916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdd083-44a9-480b-9c2e-409f1594ee5c None admin] RBAC: Adding query filter params (name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:27.584221 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdd083-44a9-480b-9c2e-409f1594ee5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:27.585630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdd083-44a9-480b-9c2e-409f1594ee5c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:27.588863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:27.597963 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 268/548] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:27.603450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdd083-44a9-480b-9c2e-409f1594ee5c None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:27.617586 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae098f0-0858-4dfe-8b33-cac9bcc523ec None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:27.663174 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ad2b3-eed6-4d5a-b954-c7d8d70a2eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:27.664044 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ad2b3-eed6-4d5a-b954-c7d8d70a2eec None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:27.664534 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ad2b3-eed6-4d5a-b954-c7d8d70a2eec None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:27.664892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1ad2b3-eed6-4d5a-b954-c7d8d70a2eec None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:27.665691 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ad2b3-eed6-4d5a-b954-c7d8d70a2eec None admin] RBAC: Adding query filter params (name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:27.666077 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ad2b3-eed6-4d5a-b954-c7d8d70a2eec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:27.667712 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1ad2b3-eed6-4d5a-b954-c7d8d70a2eec None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:27.678270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 281/549] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:27.683971 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ad2b3-eed6-4d5a-b954-c7d8d70a2eec None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:27.749275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:27.772238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdd083-44a9-480b-9c2e-409f1594ee5c None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:27.833999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1d53b-a239-4d37-be91-15ed0ffb5e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:27.836067 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1d53b-a239-4d37-be91-15ed0ffb5e8b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:27.836429 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1d53b-a239-4d37-be91-15ed0ffb5e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:27.837538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1d53b-a239-4d37-be91-15ed0ffb5e8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:27.856550 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1d53b-a239-4d37-be91-15ed0ffb5e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:27.858801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1d53b-a239-4d37-be91-15ed0ffb5e8b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:27.874908 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1ad2b3-eed6-4d5a-b954-c7d8d70a2eec None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:27.890944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-6ec1d53b-a239-4d37-be91-15ed0ffb5e8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:27.890944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:27.890944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:27.890944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:27.890944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:27.890944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:27.890944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:27.890944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:27.890944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:27.890944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:27.890944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:27.890944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:27.890944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:27.890944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:27.890944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:27.892533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:27.893774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:27.893774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:27.893774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:27.893774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:27.895578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 269/550] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:27.901266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1d53b-a239-4d37-be91-15ed0ffb5e8b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:27.946573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecbfaa-1a55-498e-99ac-ddc96672ab5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:27.947506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecbfaa-1a55-498e-99ac-ddc96672ab5a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:27.947946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecbfaa-1a55-498e-99ac-ddc96672ab5a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:27.948573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecbfaa-1a55-498e-99ac-ddc96672ab5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:27.967447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecbfaa-1a55-498e-99ac-ddc96672ab5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:27.971028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecbfaa-1a55-498e-99ac-ddc96672ab5a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:28.001930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-02ecbfaa-1a55-498e-99ac-ddc96672ab5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:28.001930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:28.001930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:28.001930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:28.001930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:28.001930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:28.001930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:28.001930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:28.001930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:28.001930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:28.001930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:28.001930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:28.001930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:28.001930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:28.001930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:28.003094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:28.004177 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:28.004177 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:28.004177 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:28.004177 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:28.010468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 282/551] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:28.017965 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecbfaa-1a55-498e-99ac-ddc96672ab5a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:28.042845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:28.124649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecbfaa-1a55-498e-99ac-ddc96672ab5a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:28.129766 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1d53b-a239-4d37-be91-15ed0ffb5e8b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:28.161953 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e6018-a1d0-4f04-956b-f7cd660dd4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:28.162884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e6018-a1d0-4f04-956b-f7cd660dd4fd None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:28.163306 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e6018-a1d0-4f04-956b-f7cd660dd4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:28.163721 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e6018-a1d0-4f04-956b-f7cd660dd4fd None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:28.164670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e6018-a1d0-4f04-956b-f7cd660dd4fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:28.165225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e6018-a1d0-4f04-956b-f7cd660dd4fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:28.166844 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31e6018-a1d0-4f04-956b-f7cd660dd4fd None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:28.168243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:28.181415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 283/552] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:28.187755 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5392e4-54a2-4c7e-aa4c-0237319c2f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:28.188387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5392e4-54a2-4c7e-aa4c-0237319c2f38 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:28.188674 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5392e4-54a2-4c7e-aa4c-0237319c2f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:28.188950 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5392e4-54a2-4c7e-aa4c-0237319c2f38 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:28.189659 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5392e4-54a2-4c7e-aa4c-0237319c2f38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:28.189966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5392e4-54a2-4c7e-aa4c-0237319c2f38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:28.191366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5392e4-54a2-4c7e-aa4c-0237319c2f38 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:28.199916 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e6018-a1d0-4f04-956b-f7cd660dd4fd None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:28.206661 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 270/553] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:28.215462 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5392e4-54a2-4c7e-aa4c-0237319c2f38 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:28.307779 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:28.350852 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5392e4-54a2-4c7e-aa4c-0237319c2f38 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:28.402294 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31e6018-a1d0-4f04-956b-f7cd660dd4fd None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:28.407362 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265807e-5797-4f97-87b4-832ad5a9626a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:28.408040 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265807e-5797-4f97-87b4-832ad5a9626a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:28.408321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265807e-5797-4f97-87b4-832ad5a9626a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:28.408598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265807e-5797-4f97-87b4-832ad5a9626a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:28.424259 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265807e-5797-4f97-87b4-832ad5a9626a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:28.425727 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265807e-5797-4f97-87b4-832ad5a9626a None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:28.447801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-4265807e-5797-4f97-87b4-832ad5a9626a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 12:40:28.447801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:28.447801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:28.447801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:28.447801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:28.447801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:28.447801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:28.447801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:28.447801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:28.447801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:28.447801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:28.447801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:28.447801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:28.447801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:28.447801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:28.447801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:28.448865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:28.449782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:28.449782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:28.449782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:28.449782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:28.449782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:28.449782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:28.449782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:28.449782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:28.449782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 12:40:28.449782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:28.454504 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 271/554] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:40:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:28.456567 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265807e-5797-4f97-87b4-832ad5a9626a None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:28.470865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e71bb2-aa6b-4bba-83cd-8428b12f3007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:28.471519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e71bb2-aa6b-4bba-83cd-8428b12f3007 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:28.471800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e71bb2-aa6b-4bba-83cd-8428b12f3007 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:28.472075 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e71bb2-aa6b-4bba-83cd-8428b12f3007 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:28.484597 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e71bb2-aa6b-4bba-83cd-8428b12f3007 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:28.486934 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e71bb2-aa6b-4bba-83cd-8428b12f3007 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:28.501310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-71e71bb2-aa6b-4bba-83cd-8428b12f3007 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:40:28.501310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:28.501310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:28.501310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:28.501310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:28.501310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:28.501310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:28.501310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:28.501310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:28.501310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:28.501310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:28.501310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:28.501310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:28.501310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:28.501310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:28.501310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:28.502311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:28.504160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:28.504160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:28.504160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:28.504160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:28.504160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:28.504160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:28.504160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:28.504160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:28.504160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:40:28.504160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:28.504160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 284/555] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 3 12:40:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:28.510697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e71bb2-aa6b-4bba-83cd-8428b12f3007 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:28.566863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:28.623670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265807e-5797-4f97-87b4-832ad5a9626a None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:28.656889 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e71bb2-aa6b-4bba-83cd-8428b12f3007 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:28.659507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6e9c5-8299-4014-a9a1-9a3d36377bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:28.660132 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6e9c5-8299-4014-a9a1-9a3d36377bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:28.660459 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6e9c5-8299-4014-a9a1-9a3d36377bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:28.660737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6e9c5-8299-4014-a9a1-9a3d36377bc3 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:28.661543 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6e9c5-8299-4014-a9a1-9a3d36377bc3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:28.661851 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6e9c5-8299-4014-a9a1-9a3d36377bc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:28.663341 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6e9c5-8299-4014-a9a1-9a3d36377bc3 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:28.681379 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:28.681940 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 272/556] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 3 12:40:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:28.687832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec1b13-573d-416e-a660-8fece282cc9c None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:28.688144 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec1b13-573d-416e-a660-8fece282cc9c None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:28.688497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec1b13-573d-416e-a660-8fece282cc9c None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:28.690215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 273/557] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:28.692076 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec1b13-573d-416e-a660-8fece282cc9c None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:28.697711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb90e13-03f6-489c-b3e9-637dbac658d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:28.698342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb90e13-03f6-489c-b3e9-637dbac658d2 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:28.698646 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb90e13-03f6-489c-b3e9-637dbac658d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:28.698915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb90e13-03f6-489c-b3e9-637dbac658d2 None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:28.699715 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb90e13-03f6-489c-b3e9-637dbac658d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:28.700024 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb90e13-03f6-489c-b3e9-637dbac658d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:28.701438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb90e13-03f6-489c-b3e9-637dbac658d2 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:28.720651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 285/558] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 3 12:40:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:28.721875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9a5ce-e14d-47e1-84cc-4c3cae3835b6 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:28.722160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9a5ce-e14d-47e1-84cc-4c3cae3835b6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:28.722561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b9a5ce-e14d-47e1-84cc-4c3cae3835b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:28.742131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-99b9a5ce-e14d-47e1-84cc-4c3cae3835b6 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:28.743604 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-99b9a5ce-e14d-47e1-84cc-4c3cae3835b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:28.801601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec1b13-573d-416e-a660-8fece282cc9c None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:28.813830 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:28.822715 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 286/559] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:28.823841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9a5ce-e14d-47e1-84cc-4c3cae3835b6 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:28.840487 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6ea80-8d3f-40b5-8875-e777ce2a989a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:28.840487 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6ea80-8d3f-40b5-8875-e777ce2a989a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:28.840487 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6ea80-8d3f-40b5-8875-e777ce2a989a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:28.840487 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6ea80-8d3f-40b5-8875-e777ce2a989a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:28.913242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6ea80-8d3f-40b5-8875-e777ce2a989a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:28.914696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6ea80-8d3f-40b5-8875-e777ce2a989a None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:28.948302 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b9a5ce-e14d-47e1-84cc-4c3cae3835b6 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:28.967161 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:28.985613 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20050956-ff3f-419c-86ef-2e6bc9bf47eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:28.986399 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20050956-ff3f-419c-86ef-2e6bc9bf47eb None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:28.986899 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20050956-ff3f-419c-86ef-2e6bc9bf47eb None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:28.987275 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20050956-ff3f-419c-86ef-2e6bc9bf47eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:28.998017 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20050956-ff3f-419c-86ef-2e6bc9bf47eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:28.999732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20050956-ff3f-419c-86ef-2e6bc9bf47eb None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:29.016364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-20050956-ff3f-419c-86ef-2e6bc9bf47eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:29.016364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:29.016364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:29.016364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:29.016364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:29.016364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:29.016364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:29.016364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:29.016364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:29.016364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:29.016364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:29.016364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:29.016364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:29.016364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:29.016364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:29.017991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:29.020180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:29.020180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:29.020180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:29.020180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:29.020180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 287/560] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:29.020180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20050956-ff3f-419c-86ef-2e6bc9bf47eb None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:29.025332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 274/561] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 12:40:29.025696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6ea80-8d3f-40b5-8875-e777ce2a989a None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:29.037550 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:29.127683 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20050956-ff3f-419c-86ef-2e6bc9bf47eb None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:29.130174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6ea80-8d3f-40b5-8875-e777ce2a989a None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:29.163890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738351bc-32a2-4a9c-be5e-84dcc50d8944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:29.164446 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738351bc-32a2-4a9c-be5e-84dcc50d8944 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:29.164721 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738351bc-32a2-4a9c-be5e-84dcc50d8944 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:29.164989 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738351bc-32a2-4a9c-be5e-84dcc50d8944 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:29.166836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd5f5e-cc1a-4b8e-aecf-2109bc380e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:29.167479 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd5f5e-cc1a-4b8e-aecf-2109bc380e0c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:29.167758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd5f5e-cc1a-4b8e-aecf-2109bc380e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:29.168030 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd5f5e-cc1a-4b8e-aecf-2109bc380e0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:29.176977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738351bc-32a2-4a9c-be5e-84dcc50d8944 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:29.178534 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd5f5e-cc1a-4b8e-aecf-2109bc380e0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:29.179987 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd5f5e-cc1a-4b8e-aecf-2109bc380e0c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:29.182599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-738351bc-32a2-4a9c-be5e-84dcc50d8944 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:29.182599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:29.182599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:29.182599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:29.182599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:29.182599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:29.182599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:29.182599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:29.182599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:29.182599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:29.182599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:29.182599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:29.182599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:29.182599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:29.182599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:29.182599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:29.183501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:29.183501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:29.183501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:29.183501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:29.183501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:29.183501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:29.183501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:29.183501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:29.183501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:29.183501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:29.183501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:29.183501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:29.183501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:29.183501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:29.183501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:29.183501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:29.189632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:29.189632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:29.189632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:29.189632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:29.189632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:29.189632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:29.189632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:29.189632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:29.189632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:29.189632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:29.189632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:29.189632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:29.189632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 288/562] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:29.189632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738351bc-32a2-4a9c-be5e-84dcc50d8944 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:29.199275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-2edd5f5e-cc1a-4b8e-aecf-2109bc380e0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:29.199275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:29.199275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:29.199275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:29.199275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:29.199275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:29.199275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:29.199275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:29.199275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:29.199275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:29.199275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:29.199275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:29.199275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:29.199275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:29.199275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:29.200191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:29.201069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:29.201069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:29.201069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:29.201069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:29.202774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 275/563] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:29.204829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea6459-b65a-4e96-bc0c-1c9aa7e1440b None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:29.205304 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea6459-b65a-4e96-bc0c-1c9aa7e1440b None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:29.205875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea6459-b65a-4e96-bc0c-1c9aa7e1440b None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:29.207951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 276/564] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:29.208548 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea6459-b65a-4e96-bc0c-1c9aa7e1440b None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:29.220151 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:29.277020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:29.302171 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738351bc-32a2-4a9c-be5e-84dcc50d8944 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:29.321409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea6459-b65a-4e96-bc0c-1c9aa7e1440b None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:29.339928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12c325-e6ab-40c2-a0e2-deff366440ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:29.340550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12c325-e6ab-40c2-a0e2-deff366440ab None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:29.341250 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12c325-e6ab-40c2-a0e2-deff366440ab None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:29.341528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12c325-e6ab-40c2-a0e2-deff366440ab None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:29.342535 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12c325-e6ab-40c2-a0e2-deff366440ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:29.342848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12c325-e6ab-40c2-a0e2-deff366440ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:29.345593 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12c325-e6ab-40c2-a0e2-deff366440ab None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:29.354748 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfa55f6-00c5-463a-8c21-f830b500c9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:29.355351 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa55f6-00c5-463a-8c21-f830b500c9db None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:29.355626 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa55f6-00c5-463a-8c21-f830b500c9db None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:29.355901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa55f6-00c5-463a-8c21-f830b500c9db None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:29.356608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa55f6-00c5-463a-8c21-f830b500c9db None admin] RBAC: Adding query filter params (name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:29.356908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa55f6-00c5-463a-8c21-f830b500c9db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:29.358276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfa55f6-00c5-463a-8c21-f830b500c9db None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:29.360332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 289/565] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:29.361653 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12c325-e6ab-40c2-a0e2-deff366440ab None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:29.367435 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 277/566] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:29.368650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee93572-24da-400e-aa92-fd9141d1bc44 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:29.368950 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee93572-24da-400e-aa92-fd9141d1bc44 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:29.369209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee93572-24da-400e-aa92-fd9141d1bc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:29.387426 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-1ee93572-24da-400e-aa92-fd9141d1bc44 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:29.388622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1ee93572-24da-400e-aa92-fd9141d1bc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:29.446013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:29.454573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 278/567] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:29.455401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee93572-24da-400e-aa92-fd9141d1bc44 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:29.462420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12c325-e6ab-40c2-a0e2-deff366440ab None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:29.497170 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01309408-a661-4d47-bca6-fe84ccdf43ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:29.497787 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01309408-a661-4d47-bca6-fe84ccdf43ed None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:29.498067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01309408-a661-4d47-bca6-fe84ccdf43ed None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:29.498345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01309408-a661-4d47-bca6-fe84ccdf43ed None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:29.499072 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01309408-a661-4d47-bca6-fe84ccdf43ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:29.499379 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01309408-a661-4d47-bca6-fe84ccdf43ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:29.500752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01309408-a661-4d47-bca6-fe84ccdf43ed None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:29.509538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:29.511847 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 290/568] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:29.513025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1c118-5361-425f-b625-d9c22e3de239 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:29.513312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1c118-5361-425f-b625-d9c22e3de239 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:29.513593 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1c118-5361-425f-b625-d9c22e3de239 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:29.514976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 291/569] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:29.516013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1c118-5361-425f-b625-d9c22e3de239 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:29.557928 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee93572-24da-400e-aa92-fd9141d1bc44 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:29.591417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fcdd9-56eb-4298-ae9c-ca2143b7e798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:29.592051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fcdd9-56eb-4298-ae9c-ca2143b7e798 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:29.592335 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fcdd9-56eb-4298-ae9c-ca2143b7e798 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:29.592605 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fcdd9-56eb-4298-ae9c-ca2143b7e798 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:29.599854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9fcdd9-56eb-4298-ae9c-ca2143b7e798 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:29.601373 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9fcdd9-56eb-4298-ae9c-ca2143b7e798 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:29.612387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-4e9fcdd9-56eb-4298-ae9c-ca2143b7e798 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:29.612387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:29.612387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:29.612387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:29.612387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:29.612387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:29.612387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:29.612387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:29.612387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:29.612387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:29.612387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:29.612387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:29.612387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:29.612387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:29.612387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:29.613279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:29.614446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:29.614446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:29.614446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:29.614446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:29.614446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:29.614446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:29.614446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:29.614446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:29.614446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 279/570] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:29.615162 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fcdd9-56eb-4298-ae9c-ca2143b7e798 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:29.622597 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1c118-5361-425f-b625-d9c22e3de239 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:29.655988 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a06181-d520-435b-90a5-8722b84c16a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:29.657222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a06181-d520-435b-90a5-8722b84c16a5 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:29.657505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a06181-d520-435b-90a5-8722b84c16a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:29.657778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a06181-d520-435b-90a5-8722b84c16a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:29.667452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a06181-d520-435b-90a5-8722b84c16a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:29.668860 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a06181-d520-435b-90a5-8722b84c16a5 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:29.671269 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:29.683992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-75a06181-d520-435b-90a5-8722b84c16a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:29.683992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:29.683992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:29.683992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:29.683992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:29.683992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:29.683992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:29.683992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:29.683992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:29.683992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:29.683992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:29.683992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:29.683992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:29.683992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:29.683992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:29.685283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:29.686172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:29.686172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:29.686172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:29.686172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:29.686172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 292/571] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:29.686172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a06181-d520-435b-90a5-8722b84c16a5 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:29.716345 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fcdd9-56eb-4298-ae9c-ca2143b7e798 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:29.730309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:29.749983 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3937c9d-b7df-46c4-b363-8e68b0278b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:29.750653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3937c9d-b7df-46c4-b363-8e68b0278b6b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:29.750938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3937c9d-b7df-46c4-b363-8e68b0278b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:29.751204 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3937c9d-b7df-46c4-b363-8e68b0278b6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:29.761015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3937c9d-b7df-46c4-b363-8e68b0278b6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:29.766458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-a3937c9d-b7df-46c4-b363-8e68b0278b6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:29.766458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:29.766458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:29.766458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:29.766458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:29.766458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:29.766458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:29.766458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:29.766458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:29.766458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:29.766458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:29.766458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:29.766458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:29.766458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:29.766458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:29.766458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:29.767409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:29.767409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:29.767409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:29.767409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:29.767409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:29.767409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:29.767409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:29.767409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:29.767409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:29.767409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:29.767409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:29.767409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:29.767409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:29.767409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:29.767409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:29.767409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:29.768267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:29.768267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:29.768267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:29.768267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:29.768267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:29.768267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:29.768267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:29.768267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:29.768267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:29.768267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:29.768267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:29.768267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:29.768267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 280/572] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:29.768967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ee895-4fdd-48d2-872c-82c39915174f None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:29.769079 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ee895-4fdd-48d2-872c-82c39915174f None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:29.769357 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ee895-4fdd-48d2-872c-82c39915174f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:29.786869 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-d45ee895-4fdd-48d2-872c-82c39915174f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:29.788055 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d45ee895-4fdd-48d2-872c-82c39915174f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:29.788461 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a06181-d520-435b-90a5-8722b84c16a5 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:29.820748 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60867c95-a0cc-44f0-bbba-367b868169bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:29.821380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60867c95-a0cc-44f0-bbba-367b868169bd None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:29.821662 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60867c95-a0cc-44f0-bbba-367b868169bd None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:29.821943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60867c95-a0cc-44f0-bbba-367b868169bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:29.828981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60867c95-a0cc-44f0-bbba-367b868169bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:29.830409 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60867c95-a0cc-44f0-bbba-367b868169bd None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:29.841515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-60867c95-a0cc-44f0-bbba-367b868169bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:29.841515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:29.841515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:29.841515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:29.841515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:29.841515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:29.841515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:29.841515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:29.841515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:29.841515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:29.841515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:29.841515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:29.841515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:29.841515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:29.841515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:29.842430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:29.843362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:29.843362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:29.843362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:29.843362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:29.843362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:29.843362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:29.843362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:29.843362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:29.843362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 293/573] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:29.844464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60867c95-a0cc-44f0-bbba-367b868169bd None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:29.856593 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 281/574] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:29.857430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ee895-4fdd-48d2-872c-82c39915174f None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:29.908360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:29.952006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60867c95-a0cc-44f0-bbba-367b868169bd None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:29.958833 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45ee895-4fdd-48d2-872c-82c39915174f None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:29.961100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:29.984775 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b9971-9018-4301-a8e0-48407adbe0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:29.985327 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b9971-9018-4301-a8e0-48407adbe0cc None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:29.985606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b9971-9018-4301-a8e0-48407adbe0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:29.985875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b9971-9018-4301-a8e0-48407adbe0cc None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:29.986654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b9971-9018-4301-a8e0-48407adbe0cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:29.986965 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b9971-9018-4301-a8e0-48407adbe0cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:29.988377 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b9971-9018-4301-a8e0-48407adbe0cc None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:29.992151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac098be-3af9-4f17-8f48-2414858bc226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:29.992763 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac098be-3af9-4f17-8f48-2414858bc226 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:29.993045 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac098be-3af9-4f17-8f48-2414858bc226 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:29.993317 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac098be-3af9-4f17-8f48-2414858bc226 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:29.994014 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac098be-3af9-4f17-8f48-2414858bc226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:29.994316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac098be-3af9-4f17-8f48-2414858bc226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:29.995695 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac098be-3af9-4f17-8f48-2414858bc226 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:30.000409 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 294/575] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:30.001182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b9971-9018-4301-a8e0-48407adbe0cc None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:30.014736 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 282/576] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:30.014736 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac098be-3af9-4f17-8f48-2414858bc226 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:30.097715 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b9971-9018-4301-a8e0-48407adbe0cc None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:30.108575 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac098be-3af9-4f17-8f48-2414858bc226 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:30.113356 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:30.131507 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd591096-d8d6-45bb-8e32-5c5d5b0f962c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:30.132122 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd591096-d8d6-45bb-8e32-5c5d5b0f962c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:30.132401 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd591096-d8d6-45bb-8e32-5c5d5b0f962c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:30.132677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd591096-d8d6-45bb-8e32-5c5d5b0f962c None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:30.133387 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd591096-d8d6-45bb-8e32-5c5d5b0f962c None admin] RBAC: Adding query filter params (name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:30.133690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd591096-d8d6-45bb-8e32-5c5d5b0f962c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:30.135119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd591096-d8d6-45bb-8e32-5c5d5b0f962c None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:30.142095 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96025e1-f627-462c-a3c8-bbd406a57045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:30.142732 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96025e1-f627-462c-a3c8-bbd406a57045 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:30.143012 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96025e1-f627-462c-a3c8-bbd406a57045 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:30.143259 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 295/577] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:30.143378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96025e1-f627-462c-a3c8-bbd406a57045 None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:30.144015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96025e1-f627-462c-a3c8-bbd406a57045 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:30.144115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd591096-d8d6-45bb-8e32-5c5d5b0f962c None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:30.144325 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96025e1-f627-462c-a3c8-bbd406a57045 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:30.145734 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96025e1-f627-462c-a3c8-bbd406a57045 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:30.157866 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 283/578] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:30.158750 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96025e1-f627-462c-a3c8-bbd406a57045 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:30.184424 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:30.243190 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd591096-d8d6-45bb-8e32-5c5d5b0f962c None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:30.261091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96025e1-f627-462c-a3c8-bbd406a57045 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:30.275865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd1e0a-96b7-4f5b-8ef3-659b667bf10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:30.276498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd1e0a-96b7-4f5b-8ef3-659b667bf10b None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:30.276782 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd1e0a-96b7-4f5b-8ef3-659b667bf10b None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:30.277057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd1e0a-96b7-4f5b-8ef3-659b667bf10b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:30.287716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd1e0a-96b7-4f5b-8ef3-659b667bf10b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:30.293228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-8abd1e0a-96b7-4f5b-8ef3-659b667bf10b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:30.293228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:30.293228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:30.293228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:30.293228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:30.293228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:30.293228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:30.293228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:30.293228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:30.293228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:30.293228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:30.293228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:30.293228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:30.293228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:30.293228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:30.293228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:30.294149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:30.294149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:30.294149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:30.294149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:30.294149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:30.294149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:30.294149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:30.294149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:30.294149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:30.294149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:30.294149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:30.294149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:30.294149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:30.294149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:30.294149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:30.294149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:30.295039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:30.295039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:30.295039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:30.295039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:30.295039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:30.295039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:30.295039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:30.295039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:30.295039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:30.295039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:30.295039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:30.295039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:30.295039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 296/579] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:30.295875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906564d-3b00-4c86-81ac-c22965c1870f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:30.295963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd1e0a-96b7-4f5b-8ef3-659b667bf10b None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:30.296042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906564d-3b00-4c86-81ac-c22965c1870f None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:30.296042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906564d-3b00-4c86-81ac-c22965c1870f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:30.296042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906564d-3b00-4c86-81ac-c22965c1870f None admin] PATH_INFO: `/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/7b634f567dca42dc95df175a4f36c1d1/roles/ac03c5aaca774e3196210f3bdb442418` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:30.317024 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906564d-3b00-4c86-81ac-c22965c1870f None admin] RBAC: Authorizing `identity:create_grant(project_id=c0774035ad0f41c986771ef4a41c7d60, user_id=7b634f567dca42dc95df175a4f36c1d1, role_id=ac03c5aaca774e3196210f3bdb442418)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:30.318463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b906564d-3b00-4c86-81ac-c22965c1870f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:30.331725 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 284/580] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:40:30 2022] PUT /identity/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/7b634f567dca42dc95df175a4f36c1d1/roles/ac03c5aaca774e3196210f3bdb442418 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:40:30.332500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906564d-3b00-4c86-81ac-c22965c1870f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:30.339245 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:30.405985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd1e0a-96b7-4f5b-8ef3-659b667bf10b None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:30.431641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:30.447665 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654adc5-1711-463c-8703-0a514b81f450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:30.448452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654adc5-1711-463c-8703-0a514b81f450 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:30.448814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654adc5-1711-463c-8703-0a514b81f450 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:30.449102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654adc5-1711-463c-8703-0a514b81f450 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:30.457606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654adc5-1711-463c-8703-0a514b81f450 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:30.459102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3654adc5-1711-463c-8703-0a514b81f450 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:30.463578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906564d-3b00-4c86-81ac-c22965c1870f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:30.472334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-3654adc5-1711-463c-8703-0a514b81f450 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 03 12:40:30.472334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:30.472334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:30.472334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:30.472334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:30.472334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:30.472334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:30.472334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:30.472334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:30.472334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:30.472334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:30.472334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:30.472334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:30.472334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:30.472334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:30.472334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:30.473380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:30.479628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:30.479628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:30.479628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:30.479628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:30.479628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:30.479628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:30.479628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:30.479628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:30.479628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 03 12:40:30.479628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:30.479628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 297/581] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:30.479628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654adc5-1711-463c-8703-0a514b81f450 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:30.506752 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df8ace-0cfd-485f-9f5b-6244a33d8463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:30.507330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df8ace-0cfd-485f-9f5b-6244a33d8463 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:30.507609 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df8ace-0cfd-485f-9f5b-6244a33d8463 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:30.507879 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df8ace-0cfd-485f-9f5b-6244a33d8463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:30.520566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df8ace-0cfd-485f-9f5b-6244a33d8463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:30.522417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df8ace-0cfd-485f-9f5b-6244a33d8463 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:30.539786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-c8df8ace-0cfd-485f-9f5b-6244a33d8463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:30.539786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:30.539786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:30.539786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:30.539786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:30.539786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:30.539786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:30.539786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:30.539786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:30.539786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:30.539786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:30.539786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:30.539786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:30.539786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:30.539786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:30.540904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:30.542164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:30.542164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:30.542164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:30.542164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:30.542164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 285/582] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:30.542164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df8ace-0cfd-485f-9f5b-6244a33d8463 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:30.587652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:30.592342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3654adc5-1711-463c-8703-0a514b81f450 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:30.628811 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af628fcb-3d3e-4c25-b471-cce0f03192c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:30.629427 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af628fcb-3d3e-4c25-b471-cce0f03192c1 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:30.629711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af628fcb-3d3e-4c25-b471-cce0f03192c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:30.629983 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af628fcb-3d3e-4c25-b471-cce0f03192c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:30.630928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af628fcb-3d3e-4c25-b471-cce0f03192c1 None admin] RBAC: Adding query filter params (role.id=ac03c5aaca774e3196210f3bdb442418, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=cea54cb772344612aec4686d484de89f) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:30.631250 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af628fcb-3d3e-4c25-b471-cce0f03192c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:30.632678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af628fcb-3d3e-4c25-b471-cce0f03192c1 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:30.638688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 298/583] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:40:30 2022] GET /identity/v3/role_assignments?user.id=cea54cb772344612aec4686d484de89f&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=ac03c5aaca774e3196210f3bdb442418 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:30.639467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af628fcb-3d3e-4c25-b471-cce0f03192c1 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:30.667523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df8ace-0cfd-485f-9f5b-6244a33d8463 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:30.697279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:30.708297 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d3511-b4e3-4974-895e-c61c56ac508c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:30.709148 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d3511-b4e3-4974-895e-c61c56ac508c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:30.709533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d3511-b4e3-4974-895e-c61c56ac508c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:30.709937 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d3511-b4e3-4974-895e-c61c56ac508c None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:30.712310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d3511-b4e3-4974-895e-c61c56ac508c None admin] RBAC: Adding query filter params (name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:30.712310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d3511-b4e3-4974-895e-c61c56ac508c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:30.712519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d3511-b4e3-4974-895e-c61c56ac508c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:30.728085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 286/584] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:30.728085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d3511-b4e3-4974-895e-c61c56ac508c None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:30.792434 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af628fcb-3d3e-4c25-b471-cce0f03192c1 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:30.831830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614282a-afd7-45d6-8148-139af1fc40bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:30.832460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614282a-afd7-45d6-8148-139af1fc40bc None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:30.832738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614282a-afd7-45d6-8148-139af1fc40bc None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:30.833009 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614282a-afd7-45d6-8148-139af1fc40bc None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:30.833884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614282a-afd7-45d6-8148-139af1fc40bc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:30.834191 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614282a-afd7-45d6-8148-139af1fc40bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:30.835659 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614282a-afd7-45d6-8148-139af1fc40bc None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:30.853898 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 299/585] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 3 12:40:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:30.854900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614282a-afd7-45d6-8148-139af1fc40bc None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:30.876530 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d3511-b4e3-4974-895e-c61c56ac508c None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:30.884856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:30.928237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11512baa-ae6f-47d9-abe1-78935acf3758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:30.929458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11512baa-ae6f-47d9-abe1-78935acf3758 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:30.930257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11512baa-ae6f-47d9-abe1-78935acf3758 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:30.930996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11512baa-ae6f-47d9-abe1-78935acf3758 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:30.932139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11512baa-ae6f-47d9-abe1-78935acf3758 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:30.932621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11512baa-ae6f-47d9-abe1-78935acf3758 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:30.935180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11512baa-ae6f-47d9-abe1-78935acf3758 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:30.949966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 287/586] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:30.951191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11512baa-ae6f-47d9-abe1-78935acf3758 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:30.975414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614282a-afd7-45d6-8148-139af1fc40bc None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:31.004592 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:31.013909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab6e77-4698-4e54-a039-39dabdd00233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:31.014590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab6e77-4698-4e54-a039-39dabdd00233 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:31.014871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab6e77-4698-4e54-a039-39dabdd00233 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:31.015140 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab6e77-4698-4e54-a039-39dabdd00233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:31.027005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab6e77-4698-4e54-a039-39dabdd00233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:31.028555 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab6e77-4698-4e54-a039-39dabdd00233 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:31.044897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-65ab6e77-4698-4e54-a039-39dabdd00233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:31.044897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:31.044897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:31.044897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:31.044897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:31.044897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:31.044897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:31.044897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:31.044897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:31.044897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:31.044897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:31.044897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:31.044897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:31.044897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:31.044897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:31.045839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:31.047279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:31.047279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:31.047279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:31.047279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:31.051291 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 300/587] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:31.052152 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab6e77-4698-4e54-a039-39dabdd00233 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:31.065829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11512baa-ae6f-47d9-abe1-78935acf3758 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:31.103116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540264a-36c3-4d93-8a3e-b1621367133c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:31.104283 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540264a-36c3-4d93-8a3e-b1621367133c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:31.104777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540264a-36c3-4d93-8a3e-b1621367133c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:31.105244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540264a-36c3-4d93-8a3e-b1621367133c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:31.117332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540264a-36c3-4d93-8a3e-b1621367133c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:31.119043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540264a-36c3-4d93-8a3e-b1621367133c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:31.124666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:31.136798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-2540264a-36c3-4d93-8a3e-b1621367133c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:31.136798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:31.136798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:31.136798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:31.136798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:31.136798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:31.136798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:31.136798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:31.136798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:31.136798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:31.136798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:31.136798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:31.136798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:31.136798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:31.136798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:31.137877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:31.139034 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:31.139034 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:31.139034 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:31.139034 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:31.140788 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 288/588] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:31.141692 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540264a-36c3-4d93-8a3e-b1621367133c None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:31.160191 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab6e77-4698-4e54-a039-39dabdd00233 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:31.203025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788a5ac-e780-46c5-93c6-cd1f254f2b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:31.203919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788a5ac-e780-46c5-93c6-cd1f254f2b09 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:31.204202 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788a5ac-e780-46c5-93c6-cd1f254f2b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:31.204477 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b788a5ac-e780-46c5-93c6-cd1f254f2b09 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:31.213078 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788a5ac-e780-46c5-93c6-cd1f254f2b09 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:31.214892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b788a5ac-e780-46c5-93c6-cd1f254f2b09 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:31.231484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-b788a5ac-e780-46c5-93c6-cd1f254f2b09 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 12:40:31.231484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:31.231484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:31.231484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:31.231484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:31.231484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:31.231484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:31.231484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:31.231484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:31.231484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:31.231484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:31.231484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:31.231484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:31.231484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:31.231484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:31.231484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:31.232442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:31.233369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:31.233369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:31.233369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:31.233369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:31.233369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:31.233369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:31.233369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:31.233369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:31.233369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 12:40:31.233369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:31.233369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 301/589] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:31.235677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788a5ac-e780-46c5-93c6-cd1f254f2b09 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:31.255663 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:31.258956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540264a-36c3-4d93-8a3e-b1621367133c None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:31.295352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd68eb9-bd75-432d-99e2-5a32eab875d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:31.295927 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd68eb9-bd75-432d-99e2-5a32eab875d5 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:31.296276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd68eb9-bd75-432d-99e2-5a32eab875d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:31.296543 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd68eb9-bd75-432d-99e2-5a32eab875d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:31.297766 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd68eb9-bd75-432d-99e2-5a32eab875d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:31.298065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd68eb9-bd75-432d-99e2-5a32eab875d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:31.299568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd68eb9-bd75-432d-99e2-5a32eab875d5 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:31.313596 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 289/590] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:31.314422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd68eb9-bd75-432d-99e2-5a32eab875d5 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:31.355160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b788a5ac-e780-46c5-93c6-cd1f254f2b09 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:31.387378 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:31.394081 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac47f65-5ad8-4ca1-90de-77d2fdfdfd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:31.394829 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac47f65-5ad8-4ca1-90de-77d2fdfdfd8f None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:31.395182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac47f65-5ad8-4ca1-90de-77d2fdfdfd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:31.395525 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac47f65-5ad8-4ca1-90de-77d2fdfdfd8f None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:31.396394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac47f65-5ad8-4ca1-90de-77d2fdfdfd8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:31.396773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac47f65-5ad8-4ca1-90de-77d2fdfdfd8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:31.398325 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac47f65-5ad8-4ca1-90de-77d2fdfdfd8f None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:31.411912 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 302/591] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:31.412857 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac47f65-5ad8-4ca1-90de-77d2fdfdfd8f None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:31.427451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd68eb9-bd75-432d-99e2-5a32eab875d5 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:31.465492 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790df214-dd06-4b66-a65a-f3fd085578a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:31.466279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790df214-dd06-4b66-a65a-f3fd085578a7 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:31.466449 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790df214-dd06-4b66-a65a-f3fd085578a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:31.466776 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790df214-dd06-4b66-a65a-f3fd085578a7 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:31.467680 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790df214-dd06-4b66-a65a-f3fd085578a7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:31.467987 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790df214-dd06-4b66-a65a-f3fd085578a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:31.469948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790df214-dd06-4b66-a65a-f3fd085578a7 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:31.493540 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 290/592] 10.222.0.65 () {56 vars in 1139 bytes} [Wed Aug 3 12:40:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:31.494429 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790df214-dd06-4b66-a65a-f3fd085578a7 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:31.514015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:31.522628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac47f65-5ad8-4ca1-90de-77d2fdfdfd8f None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:31.562549 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720200a-59ec-4820-bdee-1feaac82e8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:31.563429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720200a-59ec-4820-bdee-1feaac82e8ac None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:31.563892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720200a-59ec-4820-bdee-1feaac82e8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:31.564403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720200a-59ec-4820-bdee-1feaac82e8ac None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:31.572266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720200a-59ec-4820-bdee-1feaac82e8ac None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:31.573911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720200a-59ec-4820-bdee-1feaac82e8ac None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:31.588896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-4720200a-59ec-4820-bdee-1feaac82e8ac None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 12:40:31.588896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:31.588896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:31.588896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:31.588896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:31.588896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:31.588896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:31.588896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:31.588896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:31.588896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:31.588896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:31.588896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:31.588896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:31.588896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:31.588896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:31.588896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:31.590063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:31.591198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:31.591198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:31.591198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:31.591198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:31.591198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:31.591198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:31.591198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:31.591198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:31.591198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 12:40:31.591198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:31.598280 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 303/593] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:31.599328 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720200a-59ec-4820-bdee-1feaac82e8ac None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:31.599743 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790df214-dd06-4b66-a65a-f3fd085578a7 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:31.637113 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620a56e-7719-47fd-9441-7bb5f80facf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:31.637845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a56e-7719-47fd-9441-7bb5f80facf9 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:31.638191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a56e-7719-47fd-9441-7bb5f80facf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:31.638554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620a56e-7719-47fd-9441-7bb5f80facf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:31.645812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620a56e-7719-47fd-9441-7bb5f80facf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:31.647436 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620a56e-7719-47fd-9441-7bb5f80facf9 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:31.660309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:31.661562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-5620a56e-7719-47fd-9441-7bb5f80facf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:31.661562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:31.661562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:31.661562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:31.661562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:31.661562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:31.661562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:31.661562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:31.661562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:31.661562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:31.661562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:31.661562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:31.661562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:31.661562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:31.661562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:31.662433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:31.663360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:31.663360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:31.663360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:31.663360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:31.663360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:31.663360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:31.663360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:31.663360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:31.670186 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 291/594] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:31.671371 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620a56e-7719-47fd-9441-7bb5f80facf9 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:31.716771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720200a-59ec-4820-bdee-1feaac82e8ac None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:31.754067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882472c-4064-4359-ab61-ccbec3cca46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:31.754759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882472c-4064-4359-ab61-ccbec3cca46a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:31.755042 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882472c-4064-4359-ab61-ccbec3cca46a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:31.755313 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882472c-4064-4359-ab61-ccbec3cca46a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:31.756242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:31.768735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882472c-4064-4359-ab61-ccbec3cca46a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:31.770327 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882472c-4064-4359-ab61-ccbec3cca46a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:31.780299 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620a56e-7719-47fd-9441-7bb5f80facf9 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:31.785583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-2882472c-4064-4359-ab61-ccbec3cca46a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:31.785583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:31.785583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:31.785583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:31.785583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:31.785583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:31.785583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:31.785583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:31.785583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:31.785583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:31.785583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:31.785583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:31.785583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:31.785583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:31.785583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:31.786508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:31.787425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:31.787425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:31.787425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:31.787425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:31.791426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 304/595] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:31.792287 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882472c-4064-4359-ab61-ccbec3cca46a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:31.824846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece27686-72a0-4733-8ff4-7fd76c47d726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:31.826146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece27686-72a0-4733-8ff4-7fd76c47d726 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:31.826877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece27686-72a0-4733-8ff4-7fd76c47d726 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:31.827159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece27686-72a0-4733-8ff4-7fd76c47d726 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:31.828533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece27686-72a0-4733-8ff4-7fd76c47d726 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:31.828843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece27686-72a0-4733-8ff4-7fd76c47d726 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:31.830295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece27686-72a0-4733-8ff4-7fd76c47d726 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:31.849312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 292/596] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 3 12:40:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:31.854278 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece27686-72a0-4733-8ff4-7fd76c47d726 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:31.908210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882472c-4064-4359-ab61-ccbec3cca46a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:31.911111 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:31.947668 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73811933-7886-40c2-ad4a-585fef3abe82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:31.948657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73811933-7886-40c2-ad4a-585fef3abe82 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:31.949165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73811933-7886-40c2-ad4a-585fef3abe82 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:31.949742 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73811933-7886-40c2-ad4a-585fef3abe82 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:31.950837 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73811933-7886-40c2-ad4a-585fef3abe82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:31.951143 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73811933-7886-40c2-ad4a-585fef3abe82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:31.952562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73811933-7886-40c2-ad4a-585fef3abe82 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:31.970702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 305/597] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:31.971849 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73811933-7886-40c2-ad4a-585fef3abe82 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:31.974876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece27686-72a0-4733-8ff4-7fd76c47d726 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:32.012609 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e15959-0469-43d6-bb5e-7685750617d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:32.013527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e15959-0469-43d6-bb5e-7685750617d2 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:32.014008 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e15959-0469-43d6-bb5e-7685750617d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:32.014453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e15959-0469-43d6-bb5e-7685750617d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:32.015913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e15959-0469-43d6-bb5e-7685750617d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:32.016466 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e15959-0469-43d6-bb5e-7685750617d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:32.018181 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e15959-0469-43d6-bb5e-7685750617d2 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:32.020871 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:32.032113 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 293/598] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:32.033089 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e15959-0469-43d6-bb5e-7685750617d2 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:32.082831 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73811933-7886-40c2-ad4a-585fef3abe82 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:32.118910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11082c-b270-40f9-959d-b9ad411d9db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:32.119896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11082c-b270-40f9-959d-b9ad411d9db0 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:32.120361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11082c-b270-40f9-959d-b9ad411d9db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:32.120773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11082c-b270-40f9-959d-b9ad411d9db0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:32.132382 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11082c-b270-40f9-959d-b9ad411d9db0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:32.134068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11082c-b270-40f9-959d-b9ad411d9db0 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:32.138281 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e15959-0469-43d6-bb5e-7685750617d2 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:32.152080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-6e11082c-b270-40f9-959d-b9ad411d9db0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:32.152080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:32.152080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:32.152080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:32.152080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:32.152080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:32.152080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:32.152080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:32.152080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:32.152080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:32.152080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:32.152080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:32.152080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:32.152080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:32.152080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:32.153018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:32.153907 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:32.153907 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:32.153907 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:32.153907 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:32.156841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 306/599] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:32.157897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11082c-b270-40f9-959d-b9ad411d9db0 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:32.174415 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ba4ff-7a55-407a-8827-8c60588d2382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:32.175294 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ba4ff-7a55-407a-8827-8c60588d2382 None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:32.175519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ba4ff-7a55-407a-8827-8c60588d2382 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:32.175765 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ba4ff-7a55-407a-8827-8c60588d2382 None admin] PATH_INFO: `/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/e6066e48aa644a47a889e09cfe204858/roles/ac03c5aaca774e3196210f3bdb442418` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:32.178544 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:32.192885 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ba4ff-7a55-407a-8827-8c60588d2382 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0774035ad0f41c986771ef4a41c7d60, user_id=e6066e48aa644a47a889e09cfe204858, role_id=ac03c5aaca774e3196210f3bdb442418)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:32.194610 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ba4ff-7a55-407a-8827-8c60588d2382 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:32.210832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 294/600] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:40:32 2022] PUT /identity/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/e6066e48aa644a47a889e09cfe204858/roles/ac03c5aaca774e3196210f3bdb442418 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:40:32.211650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ba4ff-7a55-407a-8827-8c60588d2382 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:32.266136 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:32.275516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11082c-b270-40f9-959d-b9ad411d9db0 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:32.315487 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa1fff-d248-40a5-bace-52a929d7604b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:32.316522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa1fff-d248-40a5-bace-52a929d7604b None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:32.317002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa1fff-d248-40a5-bace-52a929d7604b None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:32.317491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa1fff-d248-40a5-bace-52a929d7604b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:32.325782 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa1fff-d248-40a5-bace-52a929d7604b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:32.327623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa1fff-d248-40a5-bace-52a929d7604b None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:32.328588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ba4ff-7a55-407a-8827-8c60588d2382 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:32.341466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-3bfa1fff-d248-40a5-bace-52a929d7604b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:32.341466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:32.341466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:32.341466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:32.341466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:32.341466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:32.341466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:32.341466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:32.341466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:32.341466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:32.341466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:32.341466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:32.341466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:32.341466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:32.341466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:32.342393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:32.343346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:32.343346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:32.343346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:32.343346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:32.343346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:32.343346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:32.343346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:32.343346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:32.346248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 307/601] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:32.351731 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa1fff-d248-40a5-bace-52a929d7604b None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:32.365602 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7131c-736f-45c4-9dae-dc495255289a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:32.366233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7131c-736f-45c4-9dae-dc495255289a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:32.366529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7131c-736f-45c4-9dae-dc495255289a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:32.366804 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7131c-736f-45c4-9dae-dc495255289a None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:32.367853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7131c-736f-45c4-9dae-dc495255289a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:32.367988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7131c-736f-45c4-9dae-dc495255289a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:32.369286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c7131c-736f-45c4-9dae-dc495255289a None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:32.383083 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 295/602] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:32.388356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7131c-736f-45c4-9dae-dc495255289a None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:32.460738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:32.496188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa1fff-d248-40a5-bace-52a929d7604b None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:32.508105 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7131c-736f-45c4-9dae-dc495255289a None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:32.517132 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:32.543745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77e3d6-f8cb-414b-9f38-bbb654765e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:32.544400 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77e3d6-f8cb-414b-9f38-bbb654765e52 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:32.544639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77e3d6-f8cb-414b-9f38-bbb654765e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:32.544914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77e3d6-f8cb-414b-9f38-bbb654765e52 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:32.545658 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77e3d6-f8cb-414b-9f38-bbb654765e52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:32.545966 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77e3d6-f8cb-414b-9f38-bbb654765e52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:32.548018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77e3d6-f8cb-414b-9f38-bbb654765e52 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:32.551965 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d535ef-fda3-49cc-a89e-e36eb65c7d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:32.552892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d535ef-fda3-49cc-a89e-e36eb65c7d2f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:32.553379 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d535ef-fda3-49cc-a89e-e36eb65c7d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:32.553857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d535ef-fda3-49cc-a89e-e36eb65c7d2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:32.561939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d535ef-fda3-49cc-a89e-e36eb65c7d2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:32.562135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 308/603] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:32.563955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b52878-5866-45bf-8470-7a81b9bc3876 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:32.564245 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b52878-5866-45bf-8470-7a81b9bc3876 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:32.564515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b52878-5866-45bf-8470-7a81b9bc3876 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:32.565175 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d535ef-fda3-49cc-a89e-e36eb65c7d2f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:32.569581 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 309/604] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:32.571167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2462029-c5c7-46d5-bb74-70dd7b3c6bce None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:32.571783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2462029-c5c7-46d5-bb74-70dd7b3c6bce None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:32.572013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2462029-c5c7-46d5-bb74-70dd7b3c6bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:32.581641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-32d535ef-fda3-49cc-a89e-e36eb65c7d2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:32.581641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:32.581641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:32.581641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:32.581641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:32.581641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:32.581641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:32.581641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:32.581641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:32.581641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:32.581641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:32.581641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:32.581641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:32.581641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:32.581641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:32.582698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:32.583604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:32.583604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:32.583604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:32.583604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:32.583604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:32.583604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:32.583604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:32.583604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:32.586017 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 296/605] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:32.587070 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d535ef-fda3-49cc-a89e-e36eb65c7d2f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:32.595785 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-d2462029-c5c7-46d5-bb74-70dd7b3c6bce None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:32.596929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d2462029-c5c7-46d5-bb74-70dd7b3c6bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:32.674506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 310/606] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:32.674506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2462029-c5c7-46d5-bb74-70dd7b3c6bce None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:32.731017 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d535ef-fda3-49cc-a89e-e36eb65c7d2f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:32.752333 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:32.779607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16fa3b-0755-47de-b477-79a9ed0338fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:32.781365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16fa3b-0755-47de-b477-79a9ed0338fc None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:32.782116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16fa3b-0755-47de-b477-79a9ed0338fc None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:32.782640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16fa3b-0755-47de-b477-79a9ed0338fc None admin] PATH_INFO: `/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/52ff458fd7f74b99a745edbcf13a114b/roles/ac03c5aaca774e3196210f3bdb442418` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:32.794499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2462029-c5c7-46d5-bb74-70dd7b3c6bce None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:32.801739 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16fa3b-0755-47de-b477-79a9ed0338fc None admin] RBAC: Authorizing `identity:create_grant(project_id=c0774035ad0f41c986771ef4a41c7d60, user_id=52ff458fd7f74b99a745edbcf13a114b, role_id=ac03c5aaca774e3196210f3bdb442418)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:32.803550 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16fa3b-0755-47de-b477-79a9ed0338fc None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:32.827133 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 297/607] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:40:32 2022] PUT /identity/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/52ff458fd7f74b99a745edbcf13a114b/roles/ac03c5aaca774e3196210f3bdb442418 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:40:32.827133 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16fa3b-0755-47de-b477-79a9ed0338fc None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:32.858579 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f37a70-546f-4afc-82a6-9b8f1fedf794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:32.858765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f37a70-546f-4afc-82a6-9b8f1fedf794 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:32.858987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f37a70-546f-4afc-82a6-9b8f1fedf794 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:32.859785 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f37a70-546f-4afc-82a6-9b8f1fedf794 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:32.860566 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f37a70-546f-4afc-82a6-9b8f1fedf794 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:32.866530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f37a70-546f-4afc-82a6-9b8f1fedf794 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:32.866714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f37a70-546f-4afc-82a6-9b8f1fedf794 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:32.869898 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:32.887922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 311/608] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:32.895656 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f37a70-546f-4afc-82a6-9b8f1fedf794 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:33.011677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16fa3b-0755-47de-b477-79a9ed0338fc None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:33.025332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f37a70-546f-4afc-82a6-9b8f1fedf794 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:33.059335 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:33.063561 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a447b-7dde-4fca-b132-efd1c786dcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:33.064744 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a447b-7dde-4fca-b132-efd1c786dcfd None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:33.065295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a447b-7dde-4fca-b132-efd1c786dcfd None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:33.065568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a447b-7dde-4fca-b132-efd1c786dcfd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:33.069691 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72111e85-81a6-4041-aa85-236236231d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:33.070360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72111e85-81a6-4041-aa85-236236231d4e None admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:33.070940 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72111e85-81a6-4041-aa85-236236231d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:33.071227 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72111e85-81a6-4041-aa85-236236231d4e None admin] PATH_INFO: `/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/473a5e7e980b4473acec85905fa7078c/roles/ac03c5aaca774e3196210f3bdb442418` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:33.077348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a447b-7dde-4fca-b132-efd1c786dcfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:33.089588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-530a447b-7dde-4fca-b132-efd1c786dcfd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:33.089588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:33.089588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:33.089588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:33.089588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:33.089588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:33.089588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:33.089588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:33.089588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:33.089588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:33.089588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:33.089588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:33.089588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:33.089588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:33.089588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:33.089588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:33.090608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:33.090608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:33.090608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:33.090608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:33.090608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:33.090608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:33.090608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:33.090608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:33.090608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:33.090608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:33.090608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:33.090608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:33.090608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:33.090608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:33.090608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:33.090608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:33.091455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:33.091455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:33.091455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:33.091455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:33.091455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:33.091455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:33.091455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:33.091455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:33.091455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:33.091455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:33.091455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:33.091455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:33.091455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 298/609] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:33.091455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c25ee5-054e-4bfe-9f67-75ce17b12f30 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:33.092209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c25ee5-054e-4bfe-9f67-75ce17b12f30 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:33.092209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c25ee5-054e-4bfe-9f67-75ce17b12f30 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:33.092209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 299/610] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:33.092209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c25ee5-054e-4bfe-9f67-75ce17b12f30 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:33.099678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72111e85-81a6-4041-aa85-236236231d4e None admin] RBAC: Authorizing `identity:create_grant(project_id=c0774035ad0f41c986771ef4a41c7d60, user_id=473a5e7e980b4473acec85905fa7078c, role_id=ac03c5aaca774e3196210f3bdb442418)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:33.099678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72111e85-81a6-4041-aa85-236236231d4e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:33.117561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 312/611] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:40:32 2022] PUT /identity/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/473a5e7e980b4473acec85905fa7078c/roles/ac03c5aaca774e3196210f3bdb442418 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:40:33.118835 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf6c49-7646-48c9-8def-2efb4df6e77f None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:33.119122 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf6c49-7646-48c9-8def-2efb4df6e77f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:33.120000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf6c49-7646-48c9-8def-2efb4df6e77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:33.162171 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-9eaf6c49-7646-48c9-8def-2efb4df6e77f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:33.165561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-9eaf6c49-7646-48c9-8def-2efb4df6e77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:33.266527 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 313/612] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:33.270858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:33.296365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c25ee5-054e-4bfe-9f67-75ce17b12f30 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:33.298040 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf6c49-7646-48c9-8def-2efb4df6e77f None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:33.353720 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb59077-bc66-48c9-8dc6-c13de6ba58d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:33.353720 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb59077-bc66-48c9-8dc6-c13de6ba58d4 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:33.353720 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb59077-bc66-48c9-8dc6-c13de6ba58d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:33.353720 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb59077-bc66-48c9-8dc6-c13de6ba58d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:33.354139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb59077-bc66-48c9-8dc6-c13de6ba58d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:33.354139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb59077-bc66-48c9-8dc6-c13de6ba58d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:33.354139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb59077-bc66-48c9-8dc6-c13de6ba58d4 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:33.363289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 300/613] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:33.368835 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb59077-bc66-48c9-8dc6-c13de6ba58d4 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:33.429709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf6c49-7646-48c9-8def-2efb4df6e77f None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:33.440409 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:33.490866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3024c5-9ec5-498a-a43f-262d308c00bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:33.491510 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3024c5-9ec5-498a-a43f-262d308c00bb None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:33.491785 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3024c5-9ec5-498a-a43f-262d308c00bb None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:33.492051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3024c5-9ec5-498a-a43f-262d308c00bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:33.503992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3024c5-9ec5-498a-a43f-262d308c00bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:33.508994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-6d3024c5-9ec5-498a-a43f-262d308c00bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:33.508994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:33.508994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:33.508994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:33.508994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:33.508994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:33.508994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:33.508994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:33.508994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:33.508994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:33.508994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:33.508994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:33.508994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:33.508994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:33.508994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:33.508994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:33.509984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:33.509984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:33.509984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:33.509984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:33.509984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:33.509984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:33.509984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:33.509984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:33.509984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:33.509984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:33.509984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:33.509984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:33.509984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:33.509984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:33.509984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:33.509984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:33.510923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:33.510923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:33.510923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:33.510923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:33.510923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:33.510923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:33.510923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:33.510923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:33.510923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:33.510923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:33.510923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:33.510923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:33.510923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 314/614] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:33.518473 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3024c5-9ec5-498a-a43f-262d308c00bb None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:33.544611 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb59077-bc66-48c9-8dc6-c13de6ba58d4 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:33.608999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e7bbf-0df4-45d8-a1c6-10990f184c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:33.609665 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e7bbf-0df4-45d8-a1c6-10990f184c33 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:33.609940 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e7bbf-0df4-45d8-a1c6-10990f184c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:33.610209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e7bbf-0df4-45d8-a1c6-10990f184c33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:33.621144 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:33.630815 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e7bbf-0df4-45d8-a1c6-10990f184c33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:33.632280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e7bbf-0df4-45d8-a1c6-10990f184c33 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:33.672322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-6b3e7bbf-0df4-45d8-a1c6-10990f184c33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:33.672322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:33.672322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:33.672322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:33.672322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:33.672322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:33.672322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:33.672322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:33.672322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:33.672322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:33.672322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:33.672322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:33.672322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:33.672322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:33.672322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:33.674438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:33.676611 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:33.676611 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:33.676611 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:33.676611 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:33.676611 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 301/615] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:33.682683 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e7bbf-0df4-45d8-a1c6-10990f184c33 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:33.704290 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3024c5-9ec5-498a-a43f-262d308c00bb None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:33.770431 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36490468-b1f4-4a6e-a737-6ea1707abd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:33.771453 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36490468-b1f4-4a6e-a737-6ea1707abd01 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:33.771994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36490468-b1f4-4a6e-a737-6ea1707abd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:33.772471 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36490468-b1f4-4a6e-a737-6ea1707abd01 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:33.773456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36490468-b1f4-4a6e-a737-6ea1707abd01 None admin] RBAC: Adding query filter params (name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:33.774017 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36490468-b1f4-4a6e-a737-6ea1707abd01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:33.775613 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36490468-b1f4-4a6e-a737-6ea1707abd01 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:33.792623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 315/616] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:33.795851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36490468-b1f4-4a6e-a737-6ea1707abd01 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:33.852114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:33.863899 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e7bbf-0df4-45d8-a1c6-10990f184c33 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:33.931871 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4d61f-6b50-4faf-b875-fabbfc35f194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:33.932490 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4d61f-6b50-4faf-b875-fabbfc35f194 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:33.932765 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4d61f-6b50-4faf-b875-fabbfc35f194 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:33.933035 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4d61f-6b50-4faf-b875-fabbfc35f194 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:33.933728 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4d61f-6b50-4faf-b875-fabbfc35f194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:33.934023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4d61f-6b50-4faf-b875-fabbfc35f194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:33.935668 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc4d61f-6b50-4faf-b875-fabbfc35f194 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:33.963718 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 302/617] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:33.967195 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4d61f-6b50-4faf-b875-fabbfc35f194 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:33.968809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36490468-b1f4-4a6e-a737-6ea1707abd01 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:34.026311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54ccf6-b2fa-468c-bb51-bf93320fcede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:34.027300 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54ccf6-b2fa-468c-bb51-bf93320fcede None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:34.027586 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54ccf6-b2fa-468c-bb51-bf93320fcede None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:34.027867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54ccf6-b2fa-468c-bb51-bf93320fcede None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:34.039646 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54ccf6-b2fa-468c-bb51-bf93320fcede None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:34.041114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54ccf6-b2fa-468c-bb51-bf93320fcede None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:34.059167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-8b54ccf6-b2fa-468c-bb51-bf93320fcede None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:34.059167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:34.059167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:34.059167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:34.059167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:34.059167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:34.059167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:34.059167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:34.059167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:34.059167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:34.059167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:34.059167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:34.059167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:34.059167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:34.059167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:34.060454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:34.061371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:34.061371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:34.061371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:34.061371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:34.061603 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 316/618] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:34.067353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54ccf6-b2fa-468c-bb51-bf93320fcede None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:34.086850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:34.163030 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:34.182196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54ccf6-b2fa-468c-bb51-bf93320fcede None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:34.216717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4d61f-6b50-4faf-b875-fabbfc35f194 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:34.259410 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26762804-0248-4eb6-be67-2522bdccee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:34.260032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26762804-0248-4eb6-be67-2522bdccee21 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:34.260309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26762804-0248-4eb6-be67-2522bdccee21 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:34.260575 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26762804-0248-4eb6-be67-2522bdccee21 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:34.261263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26762804-0248-4eb6-be67-2522bdccee21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:34.261623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26762804-0248-4eb6-be67-2522bdccee21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:34.263005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26762804-0248-4eb6-be67-2522bdccee21 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:34.291611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 317/619] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:34.293439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e14a2b-d7bf-4f6e-aa30-10c68ce9ce6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:34.294091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e14a2b-d7bf-4f6e-aa30-10c68ce9ce6b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:34.294368 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e14a2b-d7bf-4f6e-aa30-10c68ce9ce6b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:34.294663 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e14a2b-d7bf-4f6e-aa30-10c68ce9ce6b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:34.295708 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26762804-0248-4eb6-be67-2522bdccee21 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:34.302777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e14a2b-d7bf-4f6e-aa30-10c68ce9ce6b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:34.304336 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e14a2b-d7bf-4f6e-aa30-10c68ce9ce6b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:34.322509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-89e14a2b-d7bf-4f6e-aa30-10c68ce9ce6b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:40:34.322509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:34.322509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:34.322509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:34.322509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:34.322509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:34.322509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:34.322509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:34.322509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:34.322509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:34.322509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:34.322509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:34.322509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:34.322509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:34.322509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:34.322509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:34.323497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:34.324628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:34.324628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:34.324628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:34.324628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:34.324628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:34.324628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:34.324628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:34.324628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:34.324628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:40:34.324628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:34.324628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 303/620] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 3 12:40:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:34.325233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e14a2b-d7bf-4f6e-aa30-10c68ce9ce6b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:34.454963 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e14a2b-d7bf-4f6e-aa30-10c68ce9ce6b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:34.478844 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:34.499535 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26762804-0248-4eb6-be67-2522bdccee21 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:34.505766 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3eb3b0-d452-43ca-b998-1bd71eb37433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:34.506381 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eb3b0-d452-43ca-b998-1bd71eb37433 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:34.506691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eb3b0-d452-43ca-b998-1bd71eb37433 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:34.506964 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3eb3b0-d452-43ca-b998-1bd71eb37433 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:34.507769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3eb3b0-d452-43ca-b998-1bd71eb37433 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:34.508079 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3eb3b0-d452-43ca-b998-1bd71eb37433 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:34.509547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3eb3b0-d452-43ca-b998-1bd71eb37433 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:34.533612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 304/621] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 3 12:40:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:34.535253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aca155-93a0-460f-a581-0bd339893ffb None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:34.535540 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aca155-93a0-460f-a581-0bd339893ffb None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:34.535818 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aca155-93a0-460f-a581-0bd339893ffb None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:34.541763 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 305/622] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:34.542548 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aca155-93a0-460f-a581-0bd339893ffb None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:34.545779 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed9be2-adda-49b3-ad90-ca516b3bf65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:34.548472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed9be2-adda-49b3-ad90-ca516b3bf65d None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:34.548976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed9be2-adda-49b3-ad90-ca516b3bf65d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:34.549455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed9be2-adda-49b3-ad90-ca516b3bf65d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:34.559710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed9be2-adda-49b3-ad90-ca516b3bf65d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:34.562071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed9be2-adda-49b3-ad90-ca516b3bf65d None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:34.565728 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:34.578278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-2bed9be2-adda-49b3-ad90-ca516b3bf65d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 12:40:34.578278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:34.578278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:34.578278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:34.578278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:34.578278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:34.578278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:34.578278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:34.578278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:34.578278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:34.578278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:34.578278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:34.578278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:34.578278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:34.578278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:34.578278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:34.579276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:34.580225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:34.580225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:34.580225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:34.580225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:34.580225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:34.580225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:34.580225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:34.580225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:34.580225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 12:40:34.580225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:34.583029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 318/623] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:40:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:34.587950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c35a8f-af78-462c-b9a1-0447ae67eae2 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:34.588149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c35a8f-af78-462c-b9a1-0447ae67eae2 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:34.588366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c35a8f-af78-462c-b9a1-0447ae67eae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:34.607214 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-c2c35a8f-af78-462c-b9a1-0447ae67eae2 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:34.608412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c2c35a8f-af78-462c-b9a1-0447ae67eae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:34.674468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aca155-93a0-460f-a581-0bd339893ffb None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:34.686754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 319/624] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:34.687958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c35a8f-af78-462c-b9a1-0447ae67eae2 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:34.713645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd223f-e750-4e3f-b3a6-c1c55f5f55bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:34.714864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd223f-e750-4e3f-b3a6-c1c55f5f55bb None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:34.715458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd223f-e750-4e3f-b3a6-c1c55f5f55bb None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:34.715875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd223f-e750-4e3f-b3a6-c1c55f5f55bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:34.763577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:34.806782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd223f-e750-4e3f-b3a6-c1c55f5f55bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:34.808511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd223f-e750-4e3f-b3a6-c1c55f5f55bb None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:34.816427 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c35a8f-af78-462c-b9a1-0447ae67eae2 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:34.849319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:34.857619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b105c-4883-4995-8c64-c11a5cad731c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:34.858653 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b105c-4883-4995-8c64-c11a5cad731c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:34.859247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b105c-4883-4995-8c64-c11a5cad731c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:34.859749 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b105c-4883-4995-8c64-c11a5cad731c None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:34.860942 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b105c-4883-4995-8c64-c11a5cad731c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:34.861461 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b105c-4883-4995-8c64-c11a5cad731c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:34.863223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b105c-4883-4995-8c64-c11a5cad731c None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:34.882959 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 320/625] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 3 12:40:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:34.884065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b105c-4883-4995-8c64-c11a5cad731c None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:34.941904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 306/626] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:40:34.943131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd223f-e750-4e3f-b3a6-c1c55f5f55bb None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:34.993196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b105c-4883-4995-8c64-c11a5cad731c None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:35.026590 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:35.033707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f901c-2210-46ca-828d-2ba873a1ca0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:35.034685 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f901c-2210-46ca-828d-2ba873a1ca0a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:35.035173 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f901c-2210-46ca-828d-2ba873a1ca0a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:35.035648 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f901c-2210-46ca-828d-2ba873a1ca0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:35.047445 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f901c-2210-46ca-828d-2ba873a1ca0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:35.053021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-2b7f901c-2210-46ca-828d-2ba873a1ca0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:35.053021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:35.053021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:35.053021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:35.053021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:35.053021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:35.053021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:35.053021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:35.053021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:35.053021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:35.053021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:35.053021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:35.053021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:35.053021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:35.053021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:35.053021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:35.053967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:35.053967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:35.053967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:35.053967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:35.053967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:35.053967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:35.053967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:35.053967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:35.053967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:35.053967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:35.053967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:35.053967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:35.053967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:35.053967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:35.053967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:35.053967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:35.054933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:35.054933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:35.054933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:35.054933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:35.054933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:35.054933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:35.054933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:35.054933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:35.054933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:35.054933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:35.054933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:35.054933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:35.057382 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 321/627] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:35.058474 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f901c-2210-46ca-828d-2ba873a1ca0a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:35.069965 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd223f-e750-4e3f-b3a6-c1c55f5f55bb None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:35.112863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1583cce-97fd-42cc-a3bb-f57bc89df885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:35.113995 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1583cce-97fd-42cc-a3bb-f57bc89df885 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:35.114534 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1583cce-97fd-42cc-a3bb-f57bc89df885 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:35.115060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1583cce-97fd-42cc-a3bb-f57bc89df885 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:35.117326 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:35.128492 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1583cce-97fd-42cc-a3bb-f57bc89df885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:35.130501 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1583cce-97fd-42cc-a3bb-f57bc89df885 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:35.147856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-f1583cce-97fd-42cc-a3bb-f57bc89df885 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:35.147856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:35.147856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:35.147856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:35.147856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:35.147856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:35.147856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:35.147856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:35.147856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:35.147856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:35.147856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:35.147856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:35.147856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:35.147856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:35.147856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:35.148809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:35.149714 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:35.149714 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:35.149714 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:35.149714 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:35.151574 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 307/628] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:35.156702 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1583cce-97fd-42cc-a3bb-f57bc89df885 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:35.177176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f901c-2210-46ca-828d-2ba873a1ca0a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:35.215363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0262082-d464-4aa0-8a3b-42935bc6cfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:35.216000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0262082-d464-4aa0-8a3b-42935bc6cfad None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:35.216297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0262082-d464-4aa0-8a3b-42935bc6cfad None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:35.216553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0262082-d464-4aa0-8a3b-42935bc6cfad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:35.226650 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0262082-d464-4aa0-8a3b-42935bc6cfad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:35.228089 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0262082-d464-4aa0-8a3b-42935bc6cfad None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:35.243770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-b0262082-d464-4aa0-8a3b-42935bc6cfad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:35.243770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:35.243770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:35.243770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:35.243770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:35.243770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:35.243770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:35.243770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:35.243770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:35.243770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:35.243770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:35.243770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:35.243770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:35.243770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:35.243770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:35.244686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:35.245618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:35.245618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:35.245618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:35.245618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:35.249413 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 322/629] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:35.251232 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a91a60-30d1-4689-90cc-43e85b2819fd None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:35.251522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a91a60-30d1-4689-90cc-43e85b2819fd None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:35.251799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a91a60-30d1-4689-90cc-43e85b2819fd None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:35.253189 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 323/630] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:35.253745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a91a60-30d1-4689-90cc-43e85b2819fd None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:35.264463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1583cce-97fd-42cc-a3bb-f57bc89df885 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:35.288379 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:35.305722 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a37ee1-719e-4f9e-a3e3-b35d1cc164b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:35.306345 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a37ee1-719e-4f9e-a3e3-b35d1cc164b1 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:35.306660 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a37ee1-719e-4f9e-a3e3-b35d1cc164b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:35.306933 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a37ee1-719e-4f9e-a3e3-b35d1cc164b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:35.307653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a37ee1-719e-4f9e-a3e3-b35d1cc164b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:35.307959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a37ee1-719e-4f9e-a3e3-b35d1cc164b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:35.309318 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a37ee1-719e-4f9e-a3e3-b35d1cc164b1 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:35.322393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 308/631] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:35.322393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a37ee1-719e-4f9e-a3e3-b35d1cc164b1 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:35.367387 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:35.371790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a91a60-30d1-4689-90cc-43e85b2819fd None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:35.407932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb5544-36a3-41f5-88be-def75b4e451a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:35.408542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb5544-36a3-41f5-88be-def75b4e451a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:35.408848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb5544-36a3-41f5-88be-def75b4e451a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:35.409120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb5544-36a3-41f5-88be-def75b4e451a None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:35.409810 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb5544-36a3-41f5-88be-def75b4e451a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:35.410113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb5544-36a3-41f5-88be-def75b4e451a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:35.411494 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb5544-36a3-41f5-88be-def75b4e451a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:35.422261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 324/632] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:35.423420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b1830-02a4-4bfa-be37-675f5fde612d None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:35.423703 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b1830-02a4-4bfa-be37-675f5fde612d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:35.424037 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b1830-02a4-4bfa-be37-675f5fde612d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:35.424657 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a37ee1-719e-4f9e-a3e3-b35d1cc164b1 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:35.441532 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-037b1830-02a4-4bfa-be37-675f5fde612d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:35.442754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-037b1830-02a4-4bfa-be37-675f5fde612d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:35.459730 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443137f1-1a68-42ff-b62a-154e55f62060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:35.460345 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443137f1-1a68-42ff-b62a-154e55f62060 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:35.460622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443137f1-1a68-42ff-b62a-154e55f62060 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:35.460893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443137f1-1a68-42ff-b62a-154e55f62060 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:35.461583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443137f1-1a68-42ff-b62a-154e55f62060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:35.461887 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443137f1-1a68-42ff-b62a-154e55f62060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:35.463282 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443137f1-1a68-42ff-b62a-154e55f62060 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:35.474571 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 309/633] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:35.475344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443137f1-1a68-42ff-b62a-154e55f62060 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:35.510737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 325/634] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:35.512211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d01d20-e593-4871-89b7-708d1e81d8de None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:35.512571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d01d20-e593-4871-89b7-708d1e81d8de None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:35.512917 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d01d20-e593-4871-89b7-708d1e81d8de None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:35.514165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 326/635] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:35.514934 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d01d20-e593-4871-89b7-708d1e81d8de None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:35.522756 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:35.581227 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443137f1-1a68-42ff-b62a-154e55f62060 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:35.604619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:35.615448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d01d20-e593-4871-89b7-708d1e81d8de None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:35.617069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf2011-b2c2-4a55-8ab2-ec055db88f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:35.617711 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf2011-b2c2-4a55-8ab2-ec055db88f51 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:35.617987 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf2011-b2c2-4a55-8ab2-ec055db88f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:35.618264 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf2011-b2c2-4a55-8ab2-ec055db88f51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:35.630222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf2011-b2c2-4a55-8ab2-ec055db88f51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:35.631810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf2011-b2c2-4a55-8ab2-ec055db88f51 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:35.646233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-4ecf2011-b2c2-4a55-8ab2-ec055db88f51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:35.646233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:35.646233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:35.646233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:35.646233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:35.646233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:35.646233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:35.646233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:35.646233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:35.646233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:35.646233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:35.646233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:35.646233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:35.646233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:35.646233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:35.647263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:35.648268 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:35.648268 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:35.648268 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:35.648268 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:35.648702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e7018d-c9fc-4c75-83f0-9ac1899be599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:35.649339 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7018d-c9fc-4c75-83f0-9ac1899be599 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:35.649613 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7018d-c9fc-4c75-83f0-9ac1899be599 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:35.649879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7018d-c9fc-4c75-83f0-9ac1899be599 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:35.651892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 310/636] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:35.652717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf2011-b2c2-4a55-8ab2-ec055db88f51 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:35.657187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7018d-c9fc-4c75-83f0-9ac1899be599 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:35.658629 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e7018d-c9fc-4c75-83f0-9ac1899be599 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:35.669808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-89e7018d-c9fc-4c75-83f0-9ac1899be599 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:35.669808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:35.669808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:35.669808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:35.669808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:35.669808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:35.669808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:35.669808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:35.669808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:35.669808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:35.669808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:35.669808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:35.669808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:35.669808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:35.669808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:35.670941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:35.672688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:35.672688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:35.672688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:35.672688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:35.672688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:35.672688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:35.672688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:35.672688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:35.672688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 327/637] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:35.672688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173657e-da43-4faf-b1b3-f7ce03ec45aa None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:35.673436 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173657e-da43-4faf-b1b3-f7ce03ec45aa None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:35.673436 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173657e-da43-4faf-b1b3-f7ce03ec45aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:35.690805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-2173657e-da43-4faf-b1b3-f7ce03ec45aa None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:35.691977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-2173657e-da43-4faf-b1b3-f7ce03ec45aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:35.749323 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:35.756750 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf2011-b2c2-4a55-8ab2-ec055db88f51 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:35.757542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 328/638] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:35.758352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173657e-da43-4faf-b1b3-f7ce03ec45aa None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:35.799944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b8f45-6cd5-4c9c-b06a-8928cd669b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:35.800501 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b8f45-6cd5-4c9c-b06a-8928cd669b7f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:35.800775 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b8f45-6cd5-4c9c-b06a-8928cd669b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:35.801048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b8f45-6cd5-4c9c-b06a-8928cd669b7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:35.807743 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b8f45-6cd5-4c9c-b06a-8928cd669b7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:35.809794 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b8f45-6cd5-4c9c-b06a-8928cd669b7f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:35.821002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-1f1b8f45-6cd5-4c9c-b06a-8928cd669b7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:35.821002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:35.821002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:35.821002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:35.821002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:35.821002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:35.821002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:35.821002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:35.821002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:35.821002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:35.821002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:35.821002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:35.821002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:35.821002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:35.821002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:35.822019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:35.823170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:35.823170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:35.823170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:35.823170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:35.823170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:35.823170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:35.823170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:35.823170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:35.823170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 311/639] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:35.823170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b8f45-6cd5-4c9c-b06a-8928cd669b7f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:35.839221 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:35.857348 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173657e-da43-4faf-b1b3-f7ce03ec45aa None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:35.892069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000dbbc-dd0a-4fe6-93ff-35a8bff6a38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:35.892703 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000dbbc-dd0a-4fe6-93ff-35a8bff6a38e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:35.892979 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000dbbc-dd0a-4fe6-93ff-35a8bff6a38e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:35.893248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f000dbbc-dd0a-4fe6-93ff-35a8bff6a38e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:35.902888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f000dbbc-dd0a-4fe6-93ff-35a8bff6a38e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:35.908212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-f000dbbc-dd0a-4fe6-93ff-35a8bff6a38e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:35.908212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:35.908212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:35.908212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:35.908212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:35.908212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:35.908212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:35.908212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:35.908212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:35.908212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:35.908212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:35.908212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:35.908212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:35.908212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:35.908212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:35.908212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:35.909386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:35.909386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:35.909386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:35.909386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:35.909386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:35.909386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:35.909386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:35.909386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:35.909386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:35.909386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:35.909386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:35.909386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:35.909386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:35.909386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:35.909386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:35.909386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:35.910392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:35.910392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:35.910392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:35.910392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:35.910392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:35.910392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:35.910392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:35.910392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:35.910392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:35.910392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:35.910392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:35.910392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:35.910392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 329/640] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:35.910392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000dbbc-dd0a-4fe6-93ff-35a8bff6a38e None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:35.932189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b8f45-6cd5-4c9c-b06a-8928cd669b7f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:35.964922 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5327c1-44fa-468f-9134-5dc8ec28dd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:35.965537 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5327c1-44fa-468f-9134-5dc8ec28dd9e None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:35.965818 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5327c1-44fa-468f-9134-5dc8ec28dd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:35.966093 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5327c1-44fa-468f-9134-5dc8ec28dd9e None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:35.966817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5327c1-44fa-468f-9134-5dc8ec28dd9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:35.967120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5327c1-44fa-468f-9134-5dc8ec28dd9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:35.968969 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5327c1-44fa-468f-9134-5dc8ec28dd9e None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:35.980248 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 312/641] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:35.981023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5327c1-44fa-468f-9134-5dc8ec28dd9e None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:35.996020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:36.009464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f000dbbc-dd0a-4fe6-93ff-35a8bff6a38e None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:36.042654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03547d7f-e612-4f9a-8c59-7020d5202b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:36.043262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03547d7f-e612-4f9a-8c59-7020d5202b10 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:36.043541 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03547d7f-e612-4f9a-8c59-7020d5202b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:36.043814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03547d7f-e612-4f9a-8c59-7020d5202b10 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:36.044556 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03547d7f-e612-4f9a-8c59-7020d5202b10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:36.044865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03547d7f-e612-4f9a-8c59-7020d5202b10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:36.046384 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03547d7f-e612-4f9a-8c59-7020d5202b10 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:36.055185 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:36.062633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 330/642] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:36.063605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03547d7f-e612-4f9a-8c59-7020d5202b10 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:36.080040 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5327c1-44fa-468f-9134-5dc8ec28dd9e None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:36.112911 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c053c-b27e-4a9f-b3f1-1c4bb486dc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:36.113539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c053c-b27e-4a9f-b3f1-1c4bb486dc53 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:36.113815 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c053c-b27e-4a9f-b3f1-1c4bb486dc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:36.114082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c053c-b27e-4a9f-b3f1-1c4bb486dc53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:36.124018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c053c-b27e-4a9f-b3f1-1c4bb486dc53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:36.128856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-f39c053c-b27e-4a9f-b3f1-1c4bb486dc53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:36.128856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:36.128856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:36.128856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:36.128856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:36.128856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:36.128856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:36.128856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:36.128856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:36.128856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:36.128856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:36.128856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:36.128856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:36.128856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:36.128856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:36.128856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:36.129986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:36.129986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:36.129986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:36.129986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:36.129986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:36.129986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:36.129986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:36.129986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:36.129986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:36.129986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:36.129986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:36.129986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:36.129986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:36.129986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:36.129986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:36.129986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:36.131057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:36.131057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:36.131057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:36.131057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:36.131057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:36.131057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:36.131057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:36.131057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:36.131057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:36.131057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:36.131057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:36.131057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:36.131057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 313/643] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:36.131057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c053c-b27e-4a9f-b3f1-1c4bb486dc53 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:36.165016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03547d7f-e612-4f9a-8c59-7020d5202b10 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:36.197719 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474de776-4e1b-40ac-9b86-55b7a15df247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:36.198331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474de776-4e1b-40ac-9b86-55b7a15df247 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:36.199266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474de776-4e1b-40ac-9b86-55b7a15df247 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:36.199266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474de776-4e1b-40ac-9b86-55b7a15df247 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:36.199652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474de776-4e1b-40ac-9b86-55b7a15df247 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:36.199961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474de776-4e1b-40ac-9b86-55b7a15df247 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:36.201372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474de776-4e1b-40ac-9b86-55b7a15df247 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:36.203271 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:36.213342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 331/644] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:36.214192 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474de776-4e1b-40ac-9b86-55b7a15df247 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:36.229405 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c053c-b27e-4a9f-b3f1-1c4bb486dc53 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:36.263691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d800099-70c8-42a2-8e75-2dd22fa94f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:36.264303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d800099-70c8-42a2-8e75-2dd22fa94f4a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:36.264583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d800099-70c8-42a2-8e75-2dd22fa94f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:36.264854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d800099-70c8-42a2-8e75-2dd22fa94f4a None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:36.266059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d800099-70c8-42a2-8e75-2dd22fa94f4a None admin] RBAC: Adding query filter params (name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:36.266361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d800099-70c8-42a2-8e75-2dd22fa94f4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:36.267758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d800099-70c8-42a2-8e75-2dd22fa94f4a None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:36.273659 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:36.276051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 314/645] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:36.276820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d800099-70c8-42a2-8e75-2dd22fa94f4a None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:36.315865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474de776-4e1b-40ac-9b86-55b7a15df247 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:36.351046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edb63d-34b3-4638-9db2-4b31dd437be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:36.351677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edb63d-34b3-4638-9db2-4b31dd437be5 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:36.351954 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edb63d-34b3-4638-9db2-4b31dd437be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:36.352223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08edb63d-34b3-4638-9db2-4b31dd437be5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:36.359033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08edb63d-34b3-4638-9db2-4b31dd437be5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:36.360449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08edb63d-34b3-4638-9db2-4b31dd437be5 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:36.371657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-08edb63d-34b3-4638-9db2-4b31dd437be5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 03 12:40:36.371657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:36.371657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:36.371657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:36.371657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:36.371657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:36.371657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:36.371657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:36.371657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:36.371657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:36.371657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:36.371657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:36.371657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:36.371657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:36.371657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:36.371657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:36.372786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:36.373931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:36.373931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:36.373931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:36.373931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:36.373931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:36.373931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:36.373931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:36.373931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:36.373931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 03 12:40:36.373931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:36.373931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 332/646] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:36.373931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edb63d-34b3-4638-9db2-4b31dd437be5 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:36.379689 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d800099-70c8-42a2-8e75-2dd22fa94f4a None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:36.412642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ad4a0-fff2-4de6-9dd6-82d0677647f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:36.413301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ad4a0-fff2-4de6-9dd6-82d0677647f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:36.413581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ad4a0-fff2-4de6-9dd6-82d0677647f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:36.413849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ad4a0-fff2-4de6-9dd6-82d0677647f7 None admin] PATH_INFO: `/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/cea54cb772344612aec4686d484de89f/roles/ac03c5aaca774e3196210f3bdb442418` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:36.417018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:36.428404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ad4a0-fff2-4de6-9dd6-82d0677647f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0774035ad0f41c986771ef4a41c7d60, user_id=cea54cb772344612aec4686d484de89f, role_id=ac03c5aaca774e3196210f3bdb442418)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:36.429830 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ad4a0-fff2-4de6-9dd6-82d0677647f7 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:36.442065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 315/647] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:40:36 2022] PUT /identity/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/cea54cb772344612aec4686d484de89f/roles/ac03c5aaca774e3196210f3bdb442418 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:40:36.442922 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ad4a0-fff2-4de6-9dd6-82d0677647f7 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:36.477289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08edb63d-34b3-4638-9db2-4b31dd437be5 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:36.501238 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:36.518247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbec4c-2ca5-42a9-98d4-87967ede638f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:36.519003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbec4c-2ca5-42a9-98d4-87967ede638f None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:36.519355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbec4c-2ca5-42a9-98d4-87967ede638f None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:36.519696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fbec4c-2ca5-42a9-98d4-87967ede638f None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:36.520501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbec4c-2ca5-42a9-98d4-87967ede638f None admin] RBAC: Adding query filter params (name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:36.520803 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbec4c-2ca5-42a9-98d4-87967ede638f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:36.522216 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fbec4c-2ca5-42a9-98d4-87967ede638f None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:36.536570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 333/648] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:36.536570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbec4c-2ca5-42a9-98d4-87967ede638f None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:36.574901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ad4a0-fff2-4de6-9dd6-82d0677647f7 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:36.622945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e7ba8-cfbb-495b-9a0f-23f32ceaa535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:36.623663 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e7ba8-cfbb-495b-9a0f-23f32ceaa535 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:36.624319 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e7ba8-cfbb-495b-9a0f-23f32ceaa535 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:36.624700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e7ba8-cfbb-495b-9a0f-23f32ceaa535 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:36.625635 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e7ba8-cfbb-495b-9a0f-23f32ceaa535 None admin] RBAC: Adding query filter params (role.id=ac03c5aaca774e3196210f3bdb442418, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=7b634f567dca42dc95df175a4f36c1d1) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:36.625948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e7ba8-cfbb-495b-9a0f-23f32ceaa535 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:36.627489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e7ba8-cfbb-495b-9a0f-23f32ceaa535 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:36.633772 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 316/649] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:40:36 2022] GET /identity/v3/role_assignments?user.id=7b634f567dca42dc95df175a4f36c1d1&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=ac03c5aaca774e3196210f3bdb442418 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:36.638687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e7ba8-cfbb-495b-9a0f-23f32ceaa535 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:36.658366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fbec4c-2ca5-42a9-98d4-87967ede638f None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:36.702073 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f61fd-30ed-4f0d-b631-c88722b532b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:36.703727 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f61fd-30ed-4f0d-b631-c88722b532b4 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:36.704207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f61fd-30ed-4f0d-b631-c88722b532b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:36.704651 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:36.705590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f61fd-30ed-4f0d-b631-c88722b532b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:36.722047 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f61fd-30ed-4f0d-b631-c88722b532b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:36.723805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f61fd-30ed-4f0d-b631-c88722b532b4 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:36.742709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-1c9f61fd-30ed-4f0d-b631-c88722b532b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:36.742709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:36.742709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:36.742709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:36.742709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:36.742709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:36.742709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:36.742709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:36.742709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:36.742709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:36.742709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:36.742709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:36.742709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:36.742709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:36.742709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:36.743688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:36.744565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:36.744565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:36.744565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:36.744565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:36.746600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 334/650] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:36.747667 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f61fd-30ed-4f0d-b631-c88722b532b4 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:36.772941 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e7ba8-cfbb-495b-9a0f-23f32ceaa535 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:36.789596 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:36.830703 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6e4e6-0a35-40bf-8f62-fea637b42083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:36.831283 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6e4e6-0a35-40bf-8f62-fea637b42083 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:36.831563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6e4e6-0a35-40bf-8f62-fea637b42083 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:36.831836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6e4e6-0a35-40bf-8f62-fea637b42083 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:36.832638 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6e4e6-0a35-40bf-8f62-fea637b42083 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:36.832945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6e4e6-0a35-40bf-8f62-fea637b42083 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:36.834373 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6e4e6-0a35-40bf-8f62-fea637b42083 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:36.853948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 317/651] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 3 12:40:36 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:36.854827 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6e4e6-0a35-40bf-8f62-fea637b42083 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:36.872038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f61fd-30ed-4f0d-b631-c88722b532b4 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:36.915414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740daa6f-53cb-4442-88f8-b168dbbdea64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:36.916061 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740daa6f-53cb-4442-88f8-b168dbbdea64 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:36.916337 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740daa6f-53cb-4442-88f8-b168dbbdea64 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:36.916606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740daa6f-53cb-4442-88f8-b168dbbdea64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:36.928577 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740daa6f-53cb-4442-88f8-b168dbbdea64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:36.930076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740daa6f-53cb-4442-88f8-b168dbbdea64 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:36.945932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-740daa6f-53cb-4442-88f8-b168dbbdea64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:36.945932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:36.945932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:36.945932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:36.945932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:36.945932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:36.945932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:36.945932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:36.945932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:36.945932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:36.945932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:36.945932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:36.945932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:36.945932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:36.945932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:36.947079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:36.948258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:36.948258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:36.948258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:36.948258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:36.948258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 335/652] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:36.948258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740daa6f-53cb-4442-88f8-b168dbbdea64 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:36.983257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6e4e6-0a35-40bf-8f62-fea637b42083 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:37.002848 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:37.026189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bc61e-2295-48c8-8705-b08bcbe5b008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:37.026855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bc61e-2295-48c8-8705-b08bcbe5b008 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:37.027134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bc61e-2295-48c8-8705-b08bcbe5b008 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:37.027746 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bc61e-2295-48c8-8705-b08bcbe5b008 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:37.028554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bc61e-2295-48c8-8705-b08bcbe5b008 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:37.029048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bc61e-2295-48c8-8705-b08bcbe5b008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:37.030564 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bc61e-2295-48c8-8705-b08bcbe5b008 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:37.039002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:37.044423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 318/653] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:37.045785 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bc61e-2295-48c8-8705-b08bcbe5b008 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:37.055310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740daa6f-53cb-4442-88f8-b168dbbdea64 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:37.089765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c9735-425e-44be-a9a7-d7325ec5526c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:37.090315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c9735-425e-44be-a9a7-d7325ec5526c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:37.090640 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c9735-425e-44be-a9a7-d7325ec5526c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:37.090915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c9735-425e-44be-a9a7-d7325ec5526c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:37.157062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bc61e-2295-48c8-8705-b08bcbe5b008 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:37.159737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c9735-425e-44be-a9a7-d7325ec5526c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:37.161247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993c9735-425e-44be-a9a7-d7325ec5526c None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:37.195869 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f996076-1b94-438a-bfcc-274d6e76a501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:37.196751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f996076-1b94-438a-bfcc-274d6e76a501 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:37.197127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f996076-1b94-438a-bfcc-274d6e76a501 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:37.197498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f996076-1b94-438a-bfcc-274d6e76a501 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:37.198299 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f996076-1b94-438a-bfcc-274d6e76a501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:37.198739 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f996076-1b94-438a-bfcc-274d6e76a501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:37.200199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f996076-1b94-438a-bfcc-274d6e76a501 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:37.213570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 319/654] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:37.214688 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f996076-1b94-438a-bfcc-274d6e76a501 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:37.245393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:37.251905 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:37.268603 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 336/655] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:40:37.269659 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c9735-425e-44be-a9a7-d7325ec5526c None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:37.347274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f996076-1b94-438a-bfcc-274d6e76a501 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:37.380126 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993c9735-425e-44be-a9a7-d7325ec5526c None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:37.394440 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fece0-0877-4ea5-9f5e-27f7cbaf0f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:37.395832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fece0-0877-4ea5-9f5e-27f7cbaf0f86 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:37.396579 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fece0-0877-4ea5-9f5e-27f7cbaf0f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:37.397057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fece0-0877-4ea5-9f5e-27f7cbaf0f86 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:37.407760 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fece0-0877-4ea5-9f5e-27f7cbaf0f86 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:37.409200 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fece0-0877-4ea5-9f5e-27f7cbaf0f86 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:37.419182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c03660-8cb2-4b74-b763-b64ddbd33753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:37.420112 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c03660-8cb2-4b74-b763-b64ddbd33753 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:37.420596 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c03660-8cb2-4b74-b763-b64ddbd33753 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:37.421068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c03660-8cb2-4b74-b763-b64ddbd33753 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:37.421439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-254fece0-0877-4ea5-9f5e-27f7cbaf0f86 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 12:40:37.421439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:37.421439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:37.421439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:37.421439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:37.421439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:37.421439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:37.421439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:37.421439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:37.421439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:37.421439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:37.421439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:37.421439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:37.421439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:37.421439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:37.421439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:37.422909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:37.424453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:37.424453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:37.424453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:37.424453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:37.424453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:37.424453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:37.424453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:37.424453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:37.424453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 03 12:40:37.424453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:37.424453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 320/656] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:37.424453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fece0-0877-4ea5-9f5e-27f7cbaf0f86 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:37.435075 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c03660-8cb2-4b74-b763-b64ddbd33753 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:37.437346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c03660-8cb2-4b74-b763-b64ddbd33753 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:37.453963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-64c03660-8cb2-4b74-b763-b64ddbd33753 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 12:40:37.453963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:37.453963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:37.453963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:37.453963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:37.453963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:37.453963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:37.453963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:37.453963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:37.453963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:37.453963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:37.453963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:37.453963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:37.453963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:37.453963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:37.453963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:37.455002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:37.456710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:37.456710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:37.456710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:37.456710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:37.456710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:37.456710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:37.456710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:37.456710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:37.456710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 12:40:37.456710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:37.456710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 337/657] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:37.456710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c03660-8cb2-4b74-b763-b64ddbd33753 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:37.510783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:37.537550 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:37.552917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fece0-0877-4ea5-9f5e-27f7cbaf0f86 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:37.583135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c03660-8cb2-4b74-b763-b64ddbd33753 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:37.601301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40daf06e-1f0b-4676-9050-ef8ab38a9cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:37.602914 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40daf06e-1f0b-4676-9050-ef8ab38a9cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:37.603423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40daf06e-1f0b-4676-9050-ef8ab38a9cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:37.603924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40daf06e-1f0b-4676-9050-ef8ab38a9cd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:37.619779 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40daf06e-1f0b-4676-9050-ef8ab38a9cd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:37.621236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40daf06e-1f0b-4676-9050-ef8ab38a9cd5 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:37.624116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea81be-16f9-4528-91f5-c84ee9ed0a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:37.625053 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea81be-16f9-4528-91f5-c84ee9ed0a81 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:37.625597 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea81be-16f9-4528-91f5-c84ee9ed0a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:37.626082 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdea81be-16f9-4528-91f5-c84ee9ed0a81 None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:37.627107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea81be-16f9-4528-91f5-c84ee9ed0a81 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:37.628168 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea81be-16f9-4528-91f5-c84ee9ed0a81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:37.630288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdea81be-16f9-4528-91f5-c84ee9ed0a81 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:37.636806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-40daf06e-1f0b-4676-9050-ef8ab38a9cd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:37.636806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:37.636806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:37.636806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:37.636806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:37.636806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:37.636806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:37.636806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:37.636806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:37.636806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:37.636806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:37.636806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:37.636806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:37.636806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:37.636806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:37.637721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:37.639116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:37.639116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:37.639116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:37.639116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:37.642236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 321/658] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:37.643111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40daf06e-1f0b-4676-9050-ef8ab38a9cd5 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:37.657061 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 338/659] 10.222.0.65 () {56 vars in 1139 bytes} [Wed Aug 3 12:40:37 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:37.658239 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea81be-16f9-4528-91f5-c84ee9ed0a81 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:37.767881 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40daf06e-1f0b-4676-9050-ef8ab38a9cd5 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:37.783572 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdea81be-16f9-4528-91f5-c84ee9ed0a81 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:37.786643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:37.810157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288819e-c8b6-4f0e-935c-eada31e8b7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:37.810828 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288819e-c8b6-4f0e-935c-eada31e8b7ba None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:37.811074 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288819e-c8b6-4f0e-935c-eada31e8b7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:37.811342 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288819e-c8b6-4f0e-935c-eada31e8b7ba None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:37.812281 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288819e-c8b6-4f0e-935c-eada31e8b7ba None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:37.812575 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288819e-c8b6-4f0e-935c-eada31e8b7ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:37.814002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288819e-c8b6-4f0e-935c-eada31e8b7ba None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:37.824695 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:37.832554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c41f7-d89b-410f-b745-245700e94e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:37.833443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c41f7-d89b-410f-b745-245700e94e37 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:37.833927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c41f7-d89b-410f-b745-245700e94e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:37.834524 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c41f7-d89b-410f-b745-245700e94e37 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:37.834787 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 322/660] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 3 12:40:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:37.835515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288819e-c8b6-4f0e-935c-eada31e8b7ba None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:37.836013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c41f7-d89b-410f-b745-245700e94e37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:37.836516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c41f7-d89b-410f-b745-245700e94e37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:37.838354 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c41f7-d89b-410f-b745-245700e94e37 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:37.855745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 339/661] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:37.856937 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c41f7-d89b-410f-b745-245700e94e37 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:37.960074 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288819e-c8b6-4f0e-935c-eada31e8b7ba None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:37.998760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c41f7-d89b-410f-b745-245700e94e37 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:38.003394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4819f5-7f8c-4be1-b37f-7df6de863872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:38.004021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4819f5-7f8c-4be1-b37f-7df6de863872 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:38.004302 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4819f5-7f8c-4be1-b37f-7df6de863872 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:38.004576 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4819f5-7f8c-4be1-b37f-7df6de863872 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:38.036953 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb4b36-d097-4621-9400-38d650fd6d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:38.038154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb4b36-d097-4621-9400-38d650fd6d16 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:38.038863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb4b36-d097-4621-9400-38d650fd6d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:38.039343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb4b36-d097-4621-9400-38d650fd6d16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:38.079408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:38.084541 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:38.089228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4819f5-7f8c-4be1-b37f-7df6de863872 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:38.091514 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4819f5-7f8c-4be1-b37f-7df6de863872 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:38.130359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb4b36-d097-4621-9400-38d650fd6d16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:38.131922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb4b36-d097-4621-9400-38d650fd6d16 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:38.208237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 323/662] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:40:38.209507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4819f5-7f8c-4be1-b37f-7df6de863872 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:38.238533 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 340/663] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:40:38.239273 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb4b36-d097-4621-9400-38d650fd6d16 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:38.305274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:38.316870 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:38.320969 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4819f5-7f8c-4be1-b37f-7df6de863872 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:38.348673 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb4b36-d097-4621-9400-38d650fd6d16 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:38.357846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a700f-8609-40b0-ac4e-09999dbcef1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:38.358796 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a700f-8609-40b0-ac4e-09999dbcef1d None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:38.359354 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a700f-8609-40b0-ac4e-09999dbcef1d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:38.359832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a700f-8609-40b0-ac4e-09999dbcef1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:38.367687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a700f-8609-40b0-ac4e-09999dbcef1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:38.369402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a700f-8609-40b0-ac4e-09999dbcef1d None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:38.385047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-2c2a700f-8609-40b0-ac4e-09999dbcef1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:38.385047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:38.385047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:38.385047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:38.385047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:38.385047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:38.385047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:38.385047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:38.385047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:38.385047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:38.385047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:38.385047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:38.385047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:38.385047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:38.385047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:38.385989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:38.386931 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:38.386931 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:38.386931 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:38.386931 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:38.386931 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:38.386931 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:38.386931 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:38.386931 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:38.387366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e574b4-ebf6-4456-ae3e-2c99c1060493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:38.387366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e574b4-ebf6-4456-ae3e-2c99c1060493 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:38.387366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e574b4-ebf6-4456-ae3e-2c99c1060493 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:38.387366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e574b4-ebf6-4456-ae3e-2c99c1060493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:38.390881 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 324/664] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:38.391909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a700f-8609-40b0-ac4e-09999dbcef1d None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:38.402308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e574b4-ebf6-4456-ae3e-2c99c1060493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:38.403887 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e574b4-ebf6-4456-ae3e-2c99c1060493 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:38.419910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-24e574b4-ebf6-4456-ae3e-2c99c1060493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:38.419910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:38.419910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:38.419910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:38.419910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:38.419910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:38.419910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:38.419910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:38.419910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:38.419910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:38.419910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:38.419910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:38.419910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:38.419910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:38.419910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:38.420826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:38.422396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:38.422396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:38.422396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:38.422396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:38.422396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 341/665] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:38.422396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e574b4-ebf6-4456-ae3e-2c99c1060493 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:38.523870 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a700f-8609-40b0-ac4e-09999dbcef1d None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:38.531770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e574b4-ebf6-4456-ae3e-2c99c1060493 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:38.550933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:38.560962 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5727feb-ba8c-4a15-b296-f0235aa9ade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:38.561601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5727feb-ba8c-4a15-b296-f0235aa9ade5 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:38.561899 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5727feb-ba8c-4a15-b296-f0235aa9ade5 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:38.563091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5727feb-ba8c-4a15-b296-f0235aa9ade5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:38.571032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfd96a-ad12-428a-b45e-1c3186902b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:38.571648 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfd96a-ad12-428a-b45e-1c3186902b33 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:38.571931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfd96a-ad12-428a-b45e-1c3186902b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:38.572206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfd96a-ad12-428a-b45e-1c3186902b33 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:38.572947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfd96a-ad12-428a-b45e-1c3186902b33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:38.573122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5727feb-ba8c-4a15-b296-f0235aa9ade5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:38.573279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfd96a-ad12-428a-b45e-1c3186902b33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:38.574552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5727feb-ba8c-4a15-b296-f0235aa9ade5 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:38.574668 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfd96a-ad12-428a-b45e-1c3186902b33 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:38.587732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 342/666] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:38.588882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfd96a-ad12-428a-b45e-1c3186902b33 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:38.589555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-a5727feb-ba8c-4a15-b296-f0235aa9ade5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:38.589555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:38.589555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:38.589555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:38.589555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:38.589555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:38.589555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:38.589555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:38.589555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:38.589555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:38.589555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:38.589555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:38.589555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:38.589555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:38.589555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:38.590434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:38.591923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:38.591923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:38.591923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:38.591923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:38.591923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 325/667] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:38.592340 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f761423-4e2a-4289-8ec5-710373d5b3fb None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:38.592629 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f761423-4e2a-4289-8ec5-710373d5b3fb None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:38.592910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f761423-4e2a-4289-8ec5-710373d5b3fb None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:38.594342 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 326/668] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:38.597746 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f761423-4e2a-4289-8ec5-710373d5b3fb None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:38.611005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:38.701349 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfd96a-ad12-428a-b45e-1c3186902b33 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:38.708678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f761423-4e2a-4289-8ec5-710373d5b3fb None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:38.737419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9823e-e4d5-468e-9732-da1dc02374bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:38.738172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9823e-e4d5-468e-9732-da1dc02374bd None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:38.738562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9823e-e4d5-468e-9732-da1dc02374bd None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:38.738841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9823e-e4d5-468e-9732-da1dc02374bd None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:38.739530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9823e-e4d5-468e-9732-da1dc02374bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:38.739842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9823e-e4d5-468e-9732-da1dc02374bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:38.741230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9823e-e4d5-468e-9732-da1dc02374bd None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:38.747303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b1cff-5c98-4e6f-8290-fc24684b8f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:38.748646 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b1cff-5c98-4e6f-8290-fc24684b8f50 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:38.749077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b1cff-5c98-4e6f-8290-fc24684b8f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:38.750586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b1cff-5c98-4e6f-8290-fc24684b8f50 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:38.751741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b1cff-5c98-4e6f-8290-fc24684b8f50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:38.752199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b1cff-5c98-4e6f-8290-fc24684b8f50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:38.753362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 343/669] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:38.753873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b1cff-5c98-4e6f-8290-fc24684b8f50 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:38.754175 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9823e-e4d5-468e-9732-da1dc02374bd None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:38.775562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 327/670] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:38.775562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24296d79-4e10-4259-b427-e4b3038a385c None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:38.775562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24296d79-4e10-4259-b427-e4b3038a385c None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:38.775562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24296d79-4e10-4259-b427-e4b3038a385c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:38.791383 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-24296d79-4e10-4259-b427-e4b3038a385c None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:38.792570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-24296d79-4e10-4259-b427-e4b3038a385c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:38.800464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:38.855673 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:38.861719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 328/671] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:38.862507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24296d79-4e10-4259-b427-e4b3038a385c None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:38.873935 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9823e-e4d5-468e-9732-da1dc02374bd None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:38.911340 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b58a5-4cf1-4e3f-8b05-f3dcfb420b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:38.911957 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b58a5-4cf1-4e3f-8b05-f3dcfb420b6e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:38.912237 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b58a5-4cf1-4e3f-8b05-f3dcfb420b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:38.912502 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468b58a5-4cf1-4e3f-8b05-f3dcfb420b6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:38.913286 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b58a5-4cf1-4e3f-8b05-f3dcfb420b6e None admin] RBAC: Adding query filter params (role.id=ac03c5aaca774e3196210f3bdb442418, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=e6066e48aa644a47a889e09cfe204858) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:38.913592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b58a5-4cf1-4e3f-8b05-f3dcfb420b6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:38.915055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468b58a5-4cf1-4e3f-8b05-f3dcfb420b6e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:38.920663 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 344/672] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:40:38 2022] GET /identity/v3/role_assignments?user.id=e6066e48aa644a47a889e09cfe204858&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=ac03c5aaca774e3196210f3bdb442418 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:38.921477 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b58a5-4cf1-4e3f-8b05-f3dcfb420b6e None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:38.973711 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24296d79-4e10-4259-b427-e4b3038a385c None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:39.012586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38c809-aec4-4998-9e7f-e0a15789bedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:39.013324 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38c809-aec4-4998-9e7f-e0a15789bedb None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:39.013668 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38c809-aec4-4998-9e7f-e0a15789bedb None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:39.013944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38c809-aec4-4998-9e7f-e0a15789bedb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:39.021490 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38c809-aec4-4998-9e7f-e0a15789bedb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:39.023007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38c809-aec4-4998-9e7f-e0a15789bedb None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:39.036069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-5b38c809-aec4-4998-9e7f-e0a15789bedb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:39.036069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:39.036069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:39.036069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:39.036069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:39.036069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:39.036069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:39.036069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:39.036069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:39.036069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:39.036069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:39.036069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:39.036069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:39.036069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:39.036069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:39.037065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:39.037960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:39.037960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:39.037960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:39.037960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:39.037960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:39.037960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:39.037960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:39.037960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:39.042154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 329/673] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:39.043500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324893d1-3c63-4625-a2dc-975703a2a6d3 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:39.043856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324893d1-3c63-4625-a2dc-975703a2a6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:39.044143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324893d1-3c63-4625-a2dc-975703a2a6d3 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:39.044714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:39.048700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468b58a5-4cf1-4e3f-8b05-f3dcfb420b6e None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:39.051339 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 330/674] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:39.052150 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324893d1-3c63-4625-a2dc-975703a2a6d3 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:39.090160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252b0b7-5b62-4d09-be3c-21a3859bfb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:39.090855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252b0b7-5b62-4d09-be3c-21a3859bfb46 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:39.091140 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252b0b7-5b62-4d09-be3c-21a3859bfb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:39.091415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252b0b7-5b62-4d09-be3c-21a3859bfb46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:39.099715 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252b0b7-5b62-4d09-be3c-21a3859bfb46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:39.102065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252b0b7-5b62-4d09-be3c-21a3859bfb46 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:39.114639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-8252b0b7-5b62-4d09-be3c-21a3859bfb46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:39.114639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:39.114639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:39.114639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:39.114639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:39.114639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:39.114639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:39.114639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:39.114639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:39.114639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:39.114639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:39.114639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:39.114639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:39.114639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:39.114639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:39.115539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:39.116926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:39.116926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:39.116926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:39.116926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:39.116926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:39.116926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:39.116926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:39.116926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:39.116926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 345/675] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:39.116926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252b0b7-5b62-4d09-be3c-21a3859bfb46 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:39.133305 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:39.181758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324893d1-3c63-4625-a2dc-975703a2a6d3 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:39.217519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd65ab-5d3b-430e-a004-30f9028b3f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:39.218161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd65ab-5d3b-430e-a004-30f9028b3f49 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:39.218998 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd65ab-5d3b-430e-a004-30f9028b3f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:39.219274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd65ab-5d3b-430e-a004-30f9028b3f49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:39.226589 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252b0b7-5b62-4d09-be3c-21a3859bfb46 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:39.230223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd65ab-5d3b-430e-a004-30f9028b3f49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:39.235762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-f1cd65ab-5d3b-430e-a004-30f9028b3f49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:39.235762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:39.235762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:39.235762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:39.235762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:39.235762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:39.235762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:39.235762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:39.235762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:39.235762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:39.235762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:39.235762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:39.235762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:39.235762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:39.235762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:39.235762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:39.236769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:39.236769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:39.236769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:39.236769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:39.236769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:39.236769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:39.236769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:39.236769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:39.236769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:39.236769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:39.236769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:39.236769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:39.236769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:39.236769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:39.236769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:39.236769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:39.237628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:39.237628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:39.237628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:39.237628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:39.237628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:39.237628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:39.237628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:39.237628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:39.237628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:39.237628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:39.237628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:39.237628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:39.237628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 331/676] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:39.238339 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a38404-6ff9-4d0f-aa1f-bf011eef7851 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:39.238339 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a38404-6ff9-4d0f-aa1f-bf011eef7851 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:39.239335 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a38404-6ff9-4d0f-aa1f-bf011eef7851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:39.258749 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-54a38404-6ff9-4d0f-aa1f-bf011eef7851 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:39.259937 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-54a38404-6ff9-4d0f-aa1f-bf011eef7851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:39.269565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5c9de-4bb8-4735-b759-c3d33037ef4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:39.270229 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5c9de-4bb8-4735-b759-c3d33037ef4c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:39.270548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5c9de-4bb8-4735-b759-c3d33037ef4c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:39.270833 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5c9de-4bb8-4735-b759-c3d33037ef4c None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:39.271961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5c9de-4bb8-4735-b759-c3d33037ef4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:39.272281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5c9de-4bb8-4735-b759-c3d33037ef4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:39.274049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5c9de-4bb8-4735-b759-c3d33037ef4c None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:39.287312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 346/677] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:39.292044 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5c9de-4bb8-4735-b759-c3d33037ef4c None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:39.315409 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:39.339003 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 332/678] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:39.339003 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a38404-6ff9-4d0f-aa1f-bf011eef7851 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:39.400663 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:39.403273 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5c9de-4bb8-4735-b759-c3d33037ef4c None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:39.437973 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4496a4-b762-490e-a5f7-4d1574200035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:39.438727 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4496a4-b762-490e-a5f7-4d1574200035 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:39.439007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4496a4-b762-490e-a5f7-4d1574200035 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:39.439278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4496a4-b762-490e-a5f7-4d1574200035 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:39.440028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4496a4-b762-490e-a5f7-4d1574200035 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:39.440335 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4496a4-b762-490e-a5f7-4d1574200035 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:39.441751 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4496a4-b762-490e-a5f7-4d1574200035 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:39.453449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 347/679] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:39.454229 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4496a4-b762-490e-a5f7-4d1574200035 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:39.463706 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a38404-6ff9-4d0f-aa1f-bf011eef7851 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:39.496704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefe6bd-5e0d-442c-8e67-b9ad13587ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:39.497238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefe6bd-5e0d-442c-8e67-b9ad13587ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:39.497514 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefe6bd-5e0d-442c-8e67-b9ad13587ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:39.497785 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdefe6bd-5e0d-442c-8e67-b9ad13587ed9 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:39.498552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefe6bd-5e0d-442c-8e67-b9ad13587ed9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:39.498858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefe6bd-5e0d-442c-8e67-b9ad13587ed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:39.500351 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdefe6bd-5e0d-442c-8e67-b9ad13587ed9 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:39.510524 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 333/680] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:39.510524 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefe6bd-5e0d-442c-8e67-b9ad13587ed9 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:39.555064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:39.576810 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4496a4-b762-490e-a5f7-4d1574200035 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:39.611182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2095263-275f-42a4-a639-21aa69e282ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:39.611799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2095263-275f-42a4-a639-21aa69e282ea None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:39.612079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2095263-275f-42a4-a639-21aa69e282ea None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:39.612353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2095263-275f-42a4-a639-21aa69e282ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:39.613136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2095263-275f-42a4-a639-21aa69e282ea None admin] RBAC: Adding query filter params (role.id=ac03c5aaca774e3196210f3bdb442418, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=52ff458fd7f74b99a745edbcf13a114b) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:39.613442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2095263-275f-42a4-a639-21aa69e282ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:39.614870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2095263-275f-42a4-a639-21aa69e282ea None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:39.620646 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 348/681] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:40:39 2022] GET /identity/v3/role_assignments?user.id=52ff458fd7f74b99a745edbcf13a114b&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=ac03c5aaca774e3196210f3bdb442418 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:39.621415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2095263-275f-42a4-a639-21aa69e282ea None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:39.634840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdefe6bd-5e0d-442c-8e67-b9ad13587ed9 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:39.652397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:39.674006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d0aec-464a-4cc2-a818-0cdd631be024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:39.674683 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d0aec-464a-4cc2-a818-0cdd631be024 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:39.674961 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d0aec-464a-4cc2-a818-0cdd631be024 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:39.675229 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d0aec-464a-4cc2-a818-0cdd631be024 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:39.690612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d0aec-464a-4cc2-a818-0cdd631be024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:39.696539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-fb1d0aec-464a-4cc2-a818-0cdd631be024 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:39.696539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:39.696539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:39.696539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:39.696539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:39.696539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:39.696539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:39.696539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:39.696539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:39.696539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:39.696539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:39.696539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:39.696539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:39.696539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:39.696539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:39.696539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:39.697761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:39.697761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:39.697761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:39.697761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:39.697761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:39.697761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:39.697761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:39.697761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:39.697761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:39.697761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:39.697761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:39.697761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:39.697761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:39.697761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:39.697761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:39.697761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:39.698733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:39.698733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:39.698733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:39.698733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:39.698733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:39.698733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:39.698733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:39.698733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:39.698733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:39.698733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:39.698733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:39.698733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:39.703310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 334/682] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:40:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:39.704334 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d0aec-464a-4cc2-a818-0cdd631be024 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:39.778635 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2095263-275f-42a4-a639-21aa69e282ea None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:39.821678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d0aec-464a-4cc2-a818-0cdd631be024 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:39.824203 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463fac7-31fe-4cea-884a-5affff793ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:39.825690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463fac7-31fe-4cea-884a-5affff793ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:39.826182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463fac7-31fe-4cea-884a-5affff793ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:39.826461 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463fac7-31fe-4cea-884a-5affff793ad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:39.827288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463fac7-31fe-4cea-884a-5affff793ad1 None admin] RBAC: Adding query filter params (role.id=ac03c5aaca774e3196210f3bdb442418, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=473a5e7e980b4473acec85905fa7078c) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:39.827622 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463fac7-31fe-4cea-884a-5affff793ad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:39.829586 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463fac7-31fe-4cea-884a-5affff793ad1 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:39.836525 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 349/683] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:40:39 2022] GET /identity/v3/role_assignments?user.id=473a5e7e980b4473acec85905fa7078c&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=ac03c5aaca774e3196210f3bdb442418 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:39.837840 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463fac7-31fe-4cea-884a-5affff793ad1 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:39.863894 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:39.868705 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa929443-f1aa-4d5c-9088-8b8fe4c13efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:39.869400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa929443-f1aa-4d5c-9088-8b8fe4c13efa None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:39.869654 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa929443-f1aa-4d5c-9088-8b8fe4c13efa None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:39.869927 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa929443-f1aa-4d5c-9088-8b8fe4c13efa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:39.888528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa929443-f1aa-4d5c-9088-8b8fe4c13efa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:39.890034 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa929443-f1aa-4d5c-9088-8b8fe4c13efa None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:39.906573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-aa929443-f1aa-4d5c-9088-8b8fe4c13efa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:39.906573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:39.906573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:39.906573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:39.906573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:39.906573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:39.906573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:39.906573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:39.906573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:39.906573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:39.906573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:39.906573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:39.906573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:39.906573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:39.906573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:39.907587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:39.908482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:39.908482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:39.908482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:39.908482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:39.908482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 335/684] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:39.913878 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa929443-f1aa-4d5c-9088-8b8fe4c13efa None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:39.950024 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:40.032911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463fac7-31fe-4cea-884a-5affff793ad1 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:40.076918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa929443-f1aa-4d5c-9088-8b8fe4c13efa None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:40.083253 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e2f27-875b-49c6-8e7b-c53b96ba46cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:40.083895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e2f27-875b-49c6-8e7b-c53b96ba46cc None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:40.084180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e2f27-875b-49c6-8e7b-c53b96ba46cc None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:40.084458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e2f27-875b-49c6-8e7b-c53b96ba46cc None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:40.085192 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e2f27-875b-49c6-8e7b-c53b96ba46cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:40.085498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e2f27-875b-49c6-8e7b-c53b96ba46cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:40.086927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e2f27-875b-49c6-8e7b-c53b96ba46cc None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:40.099861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 350/685] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:40.107591 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e2f27-875b-49c6-8e7b-c53b96ba46cc None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:40.121371 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfc623-31b9-49ad-8ad3-5faea991bf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:40.122080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfc623-31b9-49ad-8ad3-5faea991bf13 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:40.122403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfc623-31b9-49ad-8ad3-5faea991bf13 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:40.122754 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfc623-31b9-49ad-8ad3-5faea991bf13 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:40.123572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfc623-31b9-49ad-8ad3-5faea991bf13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:40.123877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfc623-31b9-49ad-8ad3-5faea991bf13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:40.126026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cfc623-31b9-49ad-8ad3-5faea991bf13 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:40.140768 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 336/686] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:40.151291 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfc623-31b9-49ad-8ad3-5faea991bf13 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:40.256588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:40.284425 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cfc623-31b9-49ad-8ad3-5faea991bf13 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:40.323808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:40.355196 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf7986-4e7f-4bab-bf40-aae47efc5022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:40.355858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf7986-4e7f-4bab-bf40-aae47efc5022 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:40.356146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf7986-4e7f-4bab-bf40-aae47efc5022 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:40.356424 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf7986-4e7f-4bab-bf40-aae47efc5022 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:40.359909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e2f27-875b-49c6-8e7b-c53b96ba46cc None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:40.370522 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf7986-4e7f-4bab-bf40-aae47efc5022 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:40.370522 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf7986-4e7f-4bab-bf40-aae47efc5022 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:40.386394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-3dbf7986-4e7f-4bab-bf40-aae47efc5022 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 12:40:40.386394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:40.386394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:40.386394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:40.386394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:40.386394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:40.386394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:40.386394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:40.386394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:40.386394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:40.386394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:40.386394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:40.386394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:40.386394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:40.386394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:40.386394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:40.387417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:40.388391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:40.388391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:40.388391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:40.388391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:40.388391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:40.388391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:40.388391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:40.388391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:40.388391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 12:40:40.388391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:40.390396 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 337/687] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:40:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:40.399454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf7986-4e7f-4bab-bf40-aae47efc5022 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:40.431871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72591377-e27e-4750-9353-4baba85d3613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:40.432525 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72591377-e27e-4750-9353-4baba85d3613 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:40.432807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72591377-e27e-4750-9353-4baba85d3613 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:40.433081 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72591377-e27e-4750-9353-4baba85d3613 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:40.456511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72591377-e27e-4750-9353-4baba85d3613 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:40.457970 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72591377-e27e-4750-9353-4baba85d3613 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:40.477765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-72591377-e27e-4750-9353-4baba85d3613 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:40.477765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:40.477765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:40.477765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:40.477765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:40.477765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:40.477765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:40.477765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:40.477765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:40.477765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:40.477765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:40.477765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:40.477765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:40.477765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:40.477765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:40.478919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:40.479950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:40.479950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:40.479950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:40.479950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:40.479950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 351/688] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:40.487005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72591377-e27e-4750-9353-4baba85d3613 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:40.592378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf7986-4e7f-4bab-bf40-aae47efc5022 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:40.660028 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fa2f3-b720-47e9-8e79-09f9a24e405b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:40.660679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fa2f3-b720-47e9-8e79-09f9a24e405b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:40.660956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fa2f3-b720-47e9-8e79-09f9a24e405b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:40.661226 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fa2f3-b720-47e9-8e79-09f9a24e405b None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:40.662026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fa2f3-b720-47e9-8e79-09f9a24e405b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:40.662330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fa2f3-b720-47e9-8e79-09f9a24e405b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:40.663793 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fa2f3-b720-47e9-8e79-09f9a24e405b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:40.674955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72591377-e27e-4750-9353-4baba85d3613 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:40.696876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 338/689] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 3 12:40:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:40.700145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fa2f3-b720-47e9-8e79-09f9a24e405b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:40.706773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:40.712631 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:40.737337 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f1d7d-5b2f-4ed1-8ed0-4556585f82e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:40.737966 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f1d7d-5b2f-4ed1-8ed0-4556585f82e2 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:40.738245 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f1d7d-5b2f-4ed1-8ed0-4556585f82e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:40.738534 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f1d7d-5b2f-4ed1-8ed0-4556585f82e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:40.739323 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f1d7d-5b2f-4ed1-8ed0-4556585f82e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:40.739630 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f1d7d-5b2f-4ed1-8ed0-4556585f82e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:40.740991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f1d7d-5b2f-4ed1-8ed0-4556585f82e2 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:40.762198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 352/690] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:40.766196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f1d7d-5b2f-4ed1-8ed0-4556585f82e2 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:40.878139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fa2f3-b720-47e9-8e79-09f9a24e405b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:40.941650 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f1d7d-5b2f-4ed1-8ed0-4556585f82e2 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:40.943675 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f51b8e1-106b-4194-a9f8-7c957fcf44f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:40.944299 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51b8e1-106b-4194-a9f8-7c957fcf44f0 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:40.944577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51b8e1-106b-4194-a9f8-7c957fcf44f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:40.944845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f51b8e1-106b-4194-a9f8-7c957fcf44f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:41.010457 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434d4af-2e1c-4897-bb41-edc8ccdc3b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:41.011127 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434d4af-2e1c-4897-bb41-edc8ccdc3b2e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:41.011406 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434d4af-2e1c-4897-bb41-edc8ccdc3b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:41.011677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434d4af-2e1c-4897-bb41-edc8ccdc3b2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:41.022715 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434d4af-2e1c-4897-bb41-edc8ccdc3b2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:41.024156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1434d4af-2e1c-4897-bb41-edc8ccdc3b2e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:41.043911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-1434d4af-2e1c-4897-bb41-edc8ccdc3b2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:40:41.043911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:41.043911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:41.043911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:41.043911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:41.043911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:41.043911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:41.043911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:41.043911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:41.043911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:41.043911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:41.043911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:41.043911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:41.043911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:41.043911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:41.043911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:41.044950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:41.045929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:41.045929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:41.045929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:41.045929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:41.045929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:41.045929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:41.045929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:41.045929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:41.045929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:40:41.045929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:41.046524 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 353/691] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 3 12:40:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:41.052517 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434d4af-2e1c-4897-bb41-edc8ccdc3b2e None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:41.066420 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51b8e1-106b-4194-a9f8-7c957fcf44f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:41.068023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f51b8e1-106b-4194-a9f8-7c957fcf44f0 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:41.078404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:41.118834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:41.184928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434d4af-2e1c-4897-bb41-edc8ccdc3b2e None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:41.200976 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 339/692] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 03 12:40:41.202247 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc2e99-c2f5-4015-8479-6e5cc696281c None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:41.202552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc2e99-c2f5-4015-8479-6e5cc696281c None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:41.206520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc2e99-c2f5-4015-8479-6e5cc696281c None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:41.206520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 340/693] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:41.208725 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc2e99-c2f5-4015-8479-6e5cc696281c None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:41.229215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207ee36-26a9-4f2b-a22a-7e4a911a244d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:41.230051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207ee36-26a9-4f2b-a22a-7e4a911a244d None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:41.230517 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207ee36-26a9-4f2b-a22a-7e4a911a244d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:41.230932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207ee36-26a9-4f2b-a22a-7e4a911a244d None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:41.231880 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207ee36-26a9-4f2b-a22a-7e4a911a244d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:41.232336 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207ee36-26a9-4f2b-a22a-7e4a911a244d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:41.233929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207ee36-26a9-4f2b-a22a-7e4a911a244d None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:41.252981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 354/694] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 3 12:40:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:41.254666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad82bd6-816a-4813-8b80-3ef48bbd9be6 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:41.255129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad82bd6-816a-4813-8b80-3ef48bbd9be6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:41.255568 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad82bd6-816a-4813-8b80-3ef48bbd9be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:41.275654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-0ad82bd6-816a-4813-8b80-3ef48bbd9be6 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:41.277157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0ad82bd6-816a-4813-8b80-3ef48bbd9be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:41.347538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc2e99-c2f5-4015-8479-6e5cc696281c None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:41.349518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:41.378557 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 355/695] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:41.379712 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad82bd6-816a-4813-8b80-3ef48bbd9be6 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:41.387627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4852b1b-0e6f-4a42-b7c2-5120ea55ccdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:41.388443 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4852b1b-0e6f-4a42-b7c2-5120ea55ccdf None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:41.388890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4852b1b-0e6f-4a42-b7c2-5120ea55ccdf None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:41.389326 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4852b1b-0e6f-4a42-b7c2-5120ea55ccdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:41.401013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4852b1b-0e6f-4a42-b7c2-5120ea55ccdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:41.402813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4852b1b-0e6f-4a42-b7c2-5120ea55ccdf None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:41.418308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-b4852b1b-0e6f-4a42-b7c2-5120ea55ccdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:41.418308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:41.418308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:41.418308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:41.418308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:41.418308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:41.418308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:41.418308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:41.418308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:41.418308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:41.418308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:41.418308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:41.418308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:41.418308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:41.418308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:41.419387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:41.420284 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:41.420284 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:41.420284 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:41.420284 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:41.422084 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 341/696] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:41.427480 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4852b1b-0e6f-4a42-b7c2-5120ea55ccdf None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:41.431090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:41.505939 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad82bd6-816a-4813-8b80-3ef48bbd9be6 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:41.536057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4852b1b-0e6f-4a42-b7c2-5120ea55ccdf None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:41.542097 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe3bd3-430f-4c1f-8d46-79aed731b6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:41.542876 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe3bd3-430f-4c1f-8d46-79aed731b6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:41.543259 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe3bd3-430f-4c1f-8d46-79aed731b6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:41.543634 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe3bd3-430f-4c1f-8d46-79aed731b6f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:41.574074 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69bd76-527d-45a9-a2ce-98a6a48c8723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:41.575010 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69bd76-527d-45a9-a2ce-98a6a48c8723 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:41.575436 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69bd76-527d-45a9-a2ce-98a6a48c8723 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:41.575912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69bd76-527d-45a9-a2ce-98a6a48c8723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:41.587699 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69bd76-527d-45a9-a2ce-98a6a48c8723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:41.589480 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69bd76-527d-45a9-a2ce-98a6a48c8723 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:41.604953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-be69bd76-527d-45a9-a2ce-98a6a48c8723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:41.604953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:41.604953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:41.604953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:41.604953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:41.604953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:41.604953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:41.604953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:41.604953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:41.604953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:41.604953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:41.604953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:41.604953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:41.604953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:41.604953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:41.605874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:41.606878 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:41.606878 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:41.606878 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:41.606878 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:41.609792 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 342/697] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:41.611442 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69bd76-527d-45a9-a2ce-98a6a48c8723 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:41.622851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe3bd3-430f-4c1f-8d46-79aed731b6f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:41.623429 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:41.625209 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe3bd3-430f-4c1f-8d46-79aed731b6f1 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:41.656366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:41.732413 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69bd76-527d-45a9-a2ce-98a6a48c8723 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:41.735169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 356/698] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:40:41.736657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe3bd3-430f-4c1f-8d46-79aed731b6f1 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:41.770520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01fef3-e336-4adc-95e0-ed3023863dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:41.771496 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01fef3-e336-4adc-95e0-ed3023863dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:41.771974 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01fef3-e336-4adc-95e0-ed3023863dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:41.772457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01fef3-e336-4adc-95e0-ed3023863dc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:41.773410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01fef3-e336-4adc-95e0-ed3023863dc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:41.773915 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01fef3-e336-4adc-95e0-ed3023863dc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:41.775607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb01fef3-e336-4adc-95e0-ed3023863dc2 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:41.792892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 343/699] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:41.794237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01fef3-e336-4adc-95e0-ed3023863dc2 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:41.841171 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe3bd3-430f-4c1f-8d46-79aed731b6f1 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:41.869837 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:41.872272 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:41.879081 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bd953-7c8d-43aa-9dd0-3e4e5bf9b974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:41.880047 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bd953-7c8d-43aa-9dd0-3e4e5bf9b974 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:41.880578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bd953-7c8d-43aa-9dd0-3e4e5bf9b974 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:41.881579 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bd953-7c8d-43aa-9dd0-3e4e5bf9b974 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:41.882505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bd953-7c8d-43aa-9dd0-3e4e5bf9b974 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:41.883084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bd953-7c8d-43aa-9dd0-3e4e5bf9b974 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:41.884682 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bd953-7c8d-43aa-9dd0-3e4e5bf9b974 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:41.903194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 357/700] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:41.904408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bd953-7c8d-43aa-9dd0-3e4e5bf9b974 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:41.912379 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01fef3-e336-4adc-95e0-ed3023863dc2 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:41.949264 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa931b9-71b5-4860-8b5a-2ef84294f28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:41.949764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa931b9-71b5-4860-8b5a-2ef84294f28c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:41.950039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa931b9-71b5-4860-8b5a-2ef84294f28c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:41.950310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa931b9-71b5-4860-8b5a-2ef84294f28c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:41.961083 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa931b9-71b5-4860-8b5a-2ef84294f28c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:41.962544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa931b9-71b5-4860-8b5a-2ef84294f28c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:41.978320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-8aa931b9-71b5-4860-8b5a-2ef84294f28c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:41.978320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:41.978320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:41.978320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:41.978320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:41.978320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:41.978320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:41.978320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:41.978320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:41.978320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:41.978320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:41.978320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:41.978320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:41.978320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:41.978320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:41.979397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:41.980324 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:41.980324 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:41.980324 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:41.980324 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:41.980324 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 344/701] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:41.980617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa931b9-71b5-4860-8b5a-2ef84294f28c None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:42.018906 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bd953-7c8d-43aa-9dd0-3e4e5bf9b974 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:42.058271 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba9157-984f-4143-b195-1edecaebbb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:42.058847 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba9157-984f-4143-b195-1edecaebbb63 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:42.059123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba9157-984f-4143-b195-1edecaebbb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:42.059644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba9157-984f-4143-b195-1edecaebbb63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:42.066498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba9157-984f-4143-b195-1edecaebbb63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:42.067939 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba9157-984f-4143-b195-1edecaebbb63 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:42.080315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-9bba9157-984f-4143-b195-1edecaebbb63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:42.080315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:42.080315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:42.080315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:42.080315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:42.080315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:42.080315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:42.080315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:42.080315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:42.080315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:42.080315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:42.080315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:42.080315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:42.080315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:42.080315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:42.081234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:42.082130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:42.082130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:42.082130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:42.082130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:42.082130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:42.082130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:42.082130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:42.082130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:42.082130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 358/702] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:42.082698 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba9157-984f-4143-b195-1edecaebbb63 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:42.092154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa931b9-71b5-4860-8b5a-2ef84294f28c None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:42.110574 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:42.128757 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22d067-21ea-4150-b3fb-e719de1b5c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:42.129371 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22d067-21ea-4150-b3fb-e719de1b5c14 None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:42.129648 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22d067-21ea-4150-b3fb-e719de1b5c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:42.130211 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22d067-21ea-4150-b3fb-e719de1b5c14 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:42.130617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22d067-21ea-4150-b3fb-e719de1b5c14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:42.132034 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22d067-21ea-4150-b3fb-e719de1b5c14 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:42.138962 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:42.155139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 345/703] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:40:41 2022] POST /identity/v3/users => generated 254 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:40:42.156881 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2af9ab-ec53-46e5-8f90-7a699062565c None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:42.157166 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2af9ab-ec53-46e5-8f90-7a699062565c None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:42.157435 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2af9ab-ec53-46e5-8f90-7a699062565c None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:42.163122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 346/704] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:42.163122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2af9ab-ec53-46e5-8f90-7a699062565c None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:42.206441 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba9157-984f-4143-b195-1edecaebbb63 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:42.243653 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb06139-b755-4de9-bdec-ecfddac0af26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:42.244346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb06139-b755-4de9-bdec-ecfddac0af26 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:42.244567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb06139-b755-4de9-bdec-ecfddac0af26 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:42.244827 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb06139-b755-4de9-bdec-ecfddac0af26 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:42.245569 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb06139-b755-4de9-bdec-ecfddac0af26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:42.245870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb06139-b755-4de9-bdec-ecfddac0af26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:42.247282 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb06139-b755-4de9-bdec-ecfddac0af26 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:42.260235 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 359/705] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:42.261513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25459987-5e72-4734-a849-69c30e06fad1 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:42.261791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25459987-5e72-4734-a849-69c30e06fad1 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:42.262060 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25459987-5e72-4734-a849-69c30e06fad1 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:42.266560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 360/706] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:42.266560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e803844-d74b-427e-a6db-41f9e51da19e None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:42.266560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e803844-d74b-427e-a6db-41f9e51da19e None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:42.266560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e803844-d74b-427e-a6db-41f9e51da19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:42.285023 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-4e803844-d74b-427e-a6db-41f9e51da19e None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:42.286249 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-4e803844-d74b-427e-a6db-41f9e51da19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:42.289975 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2af9ab-ec53-46e5-8f90-7a699062565c None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:42.326776 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc6b7d-5864-40a5-887c-5f88cf781b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:42.328215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc6b7d-5864-40a5-887c-5f88cf781b27 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:42.328493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc6b7d-5864-40a5-887c-5f88cf781b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:42.328764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc6b7d-5864-40a5-887c-5f88cf781b27 None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:42.329695 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc6b7d-5864-40a5-887c-5f88cf781b27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:42.329810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc6b7d-5864-40a5-887c-5f88cf781b27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:42.331313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc6b7d-5864-40a5-887c-5f88cf781b27 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:42.348076 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 347/707] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:42.348929 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc6b7d-5864-40a5-887c-5f88cf781b27 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:42.369149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 361/708] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:42.370463 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e0987-bbc2-475b-8f8c-f25f50b53a28 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:42.370780 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e0987-bbc2-475b-8f8c-f25f50b53a28 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:42.371057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e0987-bbc2-475b-8f8c-f25f50b53a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:42.391163 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-203e0987-bbc2-475b-8f8c-f25f50b53a28 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:42.392602 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-203e0987-bbc2-475b-8f8c-f25f50b53a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:42.395760 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:42.432883 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:42.478528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 362/709] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:42.478528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e0987-bbc2-475b-8f8c-f25f50b53a28 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:42.583137 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc6b7d-5864-40a5-887c-5f88cf781b27 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:42.588080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e0987-bbc2-475b-8f8c-f25f50b53a28 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:42.620756 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0150a3-5409-4c98-89f1-413af3ab0f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:42.621400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0150a3-5409-4c98-89f1-413af3ab0f90 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:42.621687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0150a3-5409-4c98-89f1-413af3ab0f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:42.621962 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0150a3-5409-4c98-89f1-413af3ab0f90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:42.623638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40665474-7c78-4589-ac97-956772c2a2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:42.624319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40665474-7c78-4589-ac97-956772c2a2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:42.624636 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40665474-7c78-4589-ac97-956772c2a2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:42.624913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40665474-7c78-4589-ac97-956772c2a2b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:42.625762 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40665474-7c78-4589-ac97-956772c2a2b1 None admin] RBAC: Adding query filter params (role.id=ac03c5aaca774e3196210f3bdb442418, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=cea54cb772344612aec4686d484de89f) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:42.626071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40665474-7c78-4589-ac97-956772c2a2b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:42.627576 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40665474-7c78-4589-ac97-956772c2a2b1 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:42.629186 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0150a3-5409-4c98-89f1-413af3ab0f90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:42.630637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0150a3-5409-4c98-89f1-413af3ab0f90 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:42.642759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 363/710] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:40:42 2022] GET /identity/v3/role_assignments?user.id=cea54cb772344612aec4686d484de89f&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=ac03c5aaca774e3196210f3bdb442418 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:42.642759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40665474-7c78-4589-ac97-956772c2a2b1 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:42.643038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-5e0150a3-5409-4c98-89f1-413af3ab0f90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:42.643038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:42.643038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:42.643038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:42.643038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:42.643038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:42.643038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:42.643038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:42.643038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:42.643038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:42.643038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:42.643038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:42.643038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:42.643038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:42.643038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:42.644047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:42.644997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:42.644997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:42.644997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:42.644997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:42.644997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:42.644997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:42.644997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:42.644997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:42.644997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 348/711] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:42.644997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0150a3-5409-4c98-89f1-413af3ab0f90 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:42.694918 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:42.767229 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:42.775934 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0150a3-5409-4c98-89f1-413af3ab0f90 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:42.795088 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40665474-7c78-4589-ac97-956772c2a2b1 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:42.817097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed488cc-f5ce-431b-ac81-8a8a3f631c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:42.817745 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed488cc-f5ce-431b-ac81-8a8a3f631c9d None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:42.818029 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed488cc-f5ce-431b-ac81-8a8a3f631c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:42.818301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed488cc-f5ce-431b-ac81-8a8a3f631c9d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:42.819080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed488cc-f5ce-431b-ac81-8a8a3f631c9d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:42.820347 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed488cc-f5ce-431b-ac81-8a8a3f631c9d None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:42.833925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-9ed488cc-f5ce-431b-ac81-8a8a3f631c9d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 12:40:42.833925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:42.833925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:40:42.833925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:40:42.833925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:40:42.833925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:40:42.833925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:40:42.833925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:40:42.833925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 12:40:42.833925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:42.833925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:40:42.833925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:42.833925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:42.833925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:42.833925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:42.833925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:42.834981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:42.834981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:42.834981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:42.834981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:42.834981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:42.834981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:42.834981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:42.834981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:40:42.834981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:40:42.834981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:40:42.834981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:40:42.834981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:42.834981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:42.834981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:42.834981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:42.834981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:42.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:42.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:42.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:42.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:42.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:42.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:42.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:42.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:42.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:42.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:42.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:42.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:40:42.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:40:42.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 12:40:42.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:42.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 349/712] 10.222.0.65 () {56 vars in 1129 bytes} [Wed Aug 3 12:40:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:42.838057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff4c86-e9e1-4001-a74c-03687de6dbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:42.838846 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff4c86-e9e1-4001-a74c-03687de6dbe3 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:42.839237 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff4c86-e9e1-4001-a74c-03687de6dbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:42.839648 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff4c86-e9e1-4001-a74c-03687de6dbe3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:42.840388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff4c86-e9e1-4001-a74c-03687de6dbe3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:42.840800 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed488cc-f5ce-431b-ac81-8a8a3f631c9d None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:42.842025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ff4c86-e9e1-4001-a74c-03687de6dbe3 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-80ff4c86-e9e1-4001-a74c-03687de6dbe3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:42.859090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:42.860262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:42.860262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:42.860262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:42.860262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:42.860262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:42.860262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:42.860262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:40:42.860262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:40:42.860262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:40:42.860262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:40:42.860262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:42.860262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:42.860262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:42.860262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:42.860262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:42.860262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:42.861268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:42.861268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:42.861268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:42.861268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:42.861268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:42.861268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:42.861268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:42.861268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:42.861268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:42.861268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:42.861268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:40:42.861268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:40:42.861268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 12:40:42.861268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:42.863331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 364/713] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:42 2022] GET /identity/v3/services/network => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:42.864164 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff4c86-e9e1-4001-a74c-03687de6dbe3 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:42.982741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed488cc-f5ce-431b-ac81-8a8a3f631c9d None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:42.984680 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:42.989198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff4c86-e9e1-4001-a74c-03687de6dbe3 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:43.025451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f486fd7-e769-4b03-818e-ea04639b9822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:43.026471 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f486fd7-e769-4b03-818e-ea04639b9822 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:43.026989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f486fd7-e769-4b03-818e-ea04639b9822 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:43.027412 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f486fd7-e769-4b03-818e-ea04639b9822 None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:43.028398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f486fd7-e769-4b03-818e-ea04639b9822 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:43.028846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f486fd7-e769-4b03-818e-ea04639b9822 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:43.028958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74b0f1-22a0-43c6-9714-9a95b11ea78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:43.029682 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74b0f1-22a0-43c6-9714-9a95b11ea78c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:43.030066 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74b0f1-22a0-43c6-9714-9a95b11ea78c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:43.030441 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74b0f1-22a0-43c6-9714-9a95b11ea78c None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:43.031022 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f486fd7-e769-4b03-818e-ea04639b9822 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:43.031394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74b0f1-22a0-43c6-9714-9a95b11ea78c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:43.031851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74b0f1-22a0-43c6-9714-9a95b11ea78c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:43.033443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74b0f1-22a0-43c6-9714-9a95b11ea78c None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:43.038685 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 365/714] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 3 12:40:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:43.039566 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74b0f1-22a0-43c6-9714-9a95b11ea78c None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:43.046812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 350/715] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:43.047832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f486fd7-e769-4b03-818e-ea04639b9822 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:43.071277 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:43.162352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74b0f1-22a0-43c6-9714-9a95b11ea78c None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:43.167772 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f486fd7-e769-4b03-818e-ea04639b9822 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:43.198645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b57b7-1d92-4a6c-993f-4e8315d3ed21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:43.199277 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b57b7-1d92-4a6c-993f-4e8315d3ed21 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:43.199567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b57b7-1d92-4a6c-993f-4e8315d3ed21 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:43.199848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b57b7-1d92-4a6c-993f-4e8315d3ed21 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:43.200564 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b57b7-1d92-4a6c-993f-4e8315d3ed21 None admin] RBAC: Adding query filter params (name=network) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:43.200831 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b57b7-1d92-4a6c-993f-4e8315d3ed21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:43.202198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b57b7-1d92-4a6c-993f-4e8315d3ed21 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:43.204736 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd312cc-56c4-4890-b05f-055b3d992d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:43.205535 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd312cc-56c4-4890-b05f-055b3d992d6c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:43.205953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd312cc-56c4-4890-b05f-055b3d992d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:43.206367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd312cc-56c4-4890-b05f-055b3d992d6c None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:43.207213 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd312cc-56c4-4890-b05f-055b3d992d6c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:43.207672 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd312cc-56c4-4890-b05f-055b3d992d6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:43.209212 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd312cc-56c4-4890-b05f-055b3d992d6c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:43.211705 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 366/716] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 3 12:40:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:43.212738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b57b7-1d92-4a6c-993f-4e8315d3ed21 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:43.215855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 351/717] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 3 12:40:43 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:43.216873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd312cc-56c4-4890-b05f-055b3d992d6c None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:43.267847 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:43.308421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:43.329370 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd312cc-56c4-4890-b05f-055b3d992d6c None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:43.402153 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe493c-24b8-42d0-87ac-db41aa07094c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:43.403110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe493c-24b8-42d0-87ac-db41aa07094c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:43.403530 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe493c-24b8-42d0-87ac-db41aa07094c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:43.405037 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe493c-24b8-42d0-87ac-db41aa07094c None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:43.406441 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe493c-24b8-42d0-87ac-db41aa07094c None admin] RBAC: Adding query filter params (type=network) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:43.407096 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe493c-24b8-42d0-87ac-db41aa07094c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:43.409924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe493c-24b8-42d0-87ac-db41aa07094c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:43.421131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b57b7-1d92-4a6c-993f-4e8315d3ed21 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:43.422108 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 352/718] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 3 12:40:43 2022] GET /identity/v3/services?type=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:43.460459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16f8b95-8566-4b16-85ee-28481cbe1614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:43.461259 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f8b95-8566-4b16-85ee-28481cbe1614 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:43.461644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f8b95-8566-4b16-85ee-28481cbe1614 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:43.462022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16f8b95-8566-4b16-85ee-28481cbe1614 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:43.463085 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f8b95-8566-4b16-85ee-28481cbe1614 None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=7b634f567dca42dc95df175a4f36c1d1) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:43.463511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f8b95-8566-4b16-85ee-28481cbe1614 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:43.465050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16f8b95-8566-4b16-85ee-28481cbe1614 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:43.472548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 367/719] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:40:43 2022] GET /identity/v3/role_assignments?user.id=7b634f567dca42dc95df175a4f36c1d1&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:44.395480 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70766b2-f390-4ee5-ada5-d42f2934a1a2 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:44.396029 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70766b2-f390-4ee5-ada5-d42f2934a1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:44.396029 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70766b2-f390-4ee5-ada5-d42f2934a1a2 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:44.397604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 353/720] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:44.402852 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011f094-1e2c-4676-864a-fce61ccc34f5 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:44.403239 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011f094-1e2c-4676-864a-fce61ccc34f5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:44.403405 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011f094-1e2c-4676-864a-fce61ccc34f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:44.421136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-9011f094-1e2c-4676-864a-fce61ccc34f5 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:44.422351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-9011f094-1e2c-4676-864a-fce61ccc34f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:44.501357 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 368/721] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 03 12:40:44.536476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70766b2-f390-4ee5-ada5-d42f2934a1a2 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:44.648385 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70766b2-f390-4ee5-ada5-d42f2934a1a2 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:44.690161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be72a5-a9de-4534-852d-ee9e0e45b164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:44.691212 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be72a5-a9de-4534-852d-ee9e0e45b164 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:44.691700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be72a5-a9de-4534-852d-ee9e0e45b164 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:44.692193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be72a5-a9de-4534-852d-ee9e0e45b164 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:44.707488 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0be72a5-a9de-4534-852d-ee9e0e45b164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:44.712632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-a0be72a5-a9de-4534-852d-ee9e0e45b164 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:44.712632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:44.712632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:44.712632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:44.712632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:44.712632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:44.712632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:44.712632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:44.712632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:44.712632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:44.712632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:44.712632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:44.712632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:44.712632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:44.712632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:44.712632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:44.713568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:44.713568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:44.713568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:44.713568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:44.713568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:44.713568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:44.713568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:44.713568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:44.713568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:44.713568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:44.713568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:44.713568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:44.713568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:44.713568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:44.713568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:44.713568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:44.714423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:44.714423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:44.714423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:44.714423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:44.714423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:44.714423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:44.714423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:44.714423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:44.714423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:44.714423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:44.714423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:44.714423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:44.716719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 354/722] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:44.722288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011f094-1e2c-4676-864a-fce61ccc34f5 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:44.758028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:44.855799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011f094-1e2c-4676-864a-fce61ccc34f5 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:44.904810 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd1676-cb36-42a2-876f-7b16cc0c23bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:44.905365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd1676-cb36-42a2-876f-7b16cc0c23bf None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:44.906176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd1676-cb36-42a2-876f-7b16cc0c23bf None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:44.906527 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd1676-cb36-42a2-876f-7b16cc0c23bf None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:44.907261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd1676-cb36-42a2-876f-7b16cc0c23bf None admin] RBAC: Adding query filter params (name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:44.907576 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd1676-cb36-42a2-876f-7b16cc0c23bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:44.908953 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd1676-cb36-42a2-876f-7b16cc0c23bf None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:44.917767 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 369/723] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:44.923585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be72a5-a9de-4534-852d-ee9e0e45b164 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:44.943296 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:45.080213 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ff994-7f75-41b3-8ed9-d182804bb096 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:45.080574 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ff994-7f75-41b3-8ed9-d182804bb096 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:45.080770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ff994-7f75-41b3-8ed9-d182804bb096 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:45.082121 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 370/724] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:40:45.083948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be72a5-a9de-4534-852d-ee9e0e45b164 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:45.087227 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026161a-0e60-4c23-be9d-49d32a5503fd None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:45.087738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026161a-0e60-4c23-be9d-49d32a5503fd None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:45.088217 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9026161a-0e60-4c23-be9d-49d32a5503fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:45.121623 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb63a5-3eed-4f78-93d0-0e85e3156b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:45.122244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb63a5-3eed-4f78-93d0-0e85e3156b20 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:45.122547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb63a5-3eed-4f78-93d0-0e85e3156b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:45.122825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb63a5-3eed-4f78-93d0-0e85e3156b20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:45.123751 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-9026161a-0e60-4c23-be9d-49d32a5503fd None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:45.125174 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-9026161a-0e60-4c23-be9d-49d32a5503fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:45.133431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb63a5-3eed-4f78-93d0-0e85e3156b20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:45.134955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb63a5-3eed-4f78-93d0-0e85e3156b20 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:45.150068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-8afb63a5-3eed-4f78-93d0-0e85e3156b20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:45.150068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:45.150068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:45.150068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:45.150068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:45.150068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:45.150068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:45.150068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:45.150068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:45.150068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:45.150068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:45.150068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:45.150068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:45.150068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:45.150068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:45.151006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:45.151918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:45.151918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:45.151918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:45.151918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:45.151918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 355/725] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:45.157727 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb63a5-3eed-4f78-93d0-0e85e3156b20 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:45.217989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:45.263496 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb63a5-3eed-4f78-93d0-0e85e3156b20 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:45.303835 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b227b-84d6-419f-b60e-e019703d761b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:45.304461 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b227b-84d6-419f-b60e-e019703d761b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:45.304739 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b227b-84d6-419f-b60e-e019703d761b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:45.305018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b227b-84d6-419f-b60e-e019703d761b None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:45.305711 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b227b-84d6-419f-b60e-e019703d761b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:45.306017 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b227b-84d6-419f-b60e-e019703d761b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:45.307409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71b227b-84d6-419f-b60e-e019703d761b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:45.312755 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 371/726] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:45.324113 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 356/727] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:45.330092 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026161a-0e60-4c23-be9d-49d32a5503fd None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:45.354864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:45.360017 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b227b-84d6-419f-b60e-e019703d761b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:45.503411 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b227b-84d6-419f-b60e-e019703d761b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:45.510519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9026161a-0e60-4c23-be9d-49d32a5503fd None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:45.522984 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:45.541860 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230431bf-594d-4db5-b757-f9e035d334f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:45.542544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230431bf-594d-4db5-b757-f9e035d334f4 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:45.542835 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230431bf-594d-4db5-b757-f9e035d334f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:45.543105 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230431bf-594d-4db5-b757-f9e035d334f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:45.554039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230431bf-594d-4db5-b757-f9e035d334f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:45.559547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-230431bf-594d-4db5-b757-f9e035d334f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:45.559547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:45.559547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:45.559547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:45.559547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:45.559547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:45.559547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:45.559547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:45.559547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:45.559547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:45.559547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:45.559547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:45.559547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:45.559547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:45.559547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:45.559547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:45.560510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:45.560510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:45.560510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:45.560510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:45.560510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:45.560510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:45.560510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:45.560510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:45.560510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:45.560510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:45.560510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:45.560510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:45.560510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:45.560510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:45.560510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:45.560510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:45.561369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:45.561369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:45.561369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:45.561369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:45.561369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:45.561369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:45.561369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:45.561369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:45.561369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:45.561369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:45.561369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:45.561369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:45.561369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 357/728] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:40:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:45.566851 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230431bf-594d-4db5-b757-f9e035d334f4 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:45.592085 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0ea98-01bc-4fc1-93dd-ccea2aba32f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:45.592749 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0ea98-01bc-4fc1-93dd-ccea2aba32f1 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:45.593036 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0ea98-01bc-4fc1-93dd-ccea2aba32f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:45.593873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0ea98-01bc-4fc1-93dd-ccea2aba32f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:45.611759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0ea98-01bc-4fc1-93dd-ccea2aba32f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:45.613418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0ea98-01bc-4fc1-93dd-ccea2aba32f1 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:45.640732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-2aa0ea98-01bc-4fc1-93dd-ccea2aba32f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 12:40:45.640732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:45.640732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:45.640732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:45.640732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:45.640732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:45.640732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:45.640732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:45.640732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:45.640732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:45.640732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:45.640732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:45.640732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:45.640732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:45.640732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:45.640732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:45.641789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:45.642799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:45.642799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:45.642799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:45.642799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:45.642799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:45.642799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:45.642799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:45.642799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:45.642799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 12:40:45.642799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:45.647157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 372/729] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 3 12:40:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:45.655458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0ea98-01bc-4fc1-93dd-ccea2aba32f1 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:45.675709 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230431bf-594d-4db5-b757-f9e035d334f4 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:45.722323 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1f33d-6bee-4353-aa68-fec74d769e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:45.723029 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1f33d-6bee-4353-aa68-fec74d769e58 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:45.723314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1f33d-6bee-4353-aa68-fec74d769e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:45.723641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1f33d-6bee-4353-aa68-fec74d769e58 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:45.724479 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1f33d-6bee-4353-aa68-fec74d769e58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:45.724788 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1f33d-6bee-4353-aa68-fec74d769e58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:45.726142 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1f33d-6bee-4353-aa68-fec74d769e58 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:45.750618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 358/730] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:45.767553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1f33d-6bee-4353-aa68-fec74d769e58 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:45.847994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:45.860257 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0ea98-01bc-4fc1-93dd-ccea2aba32f1 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:45.871373 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:45.926560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c979fd0-f02b-412d-a679-060c40389edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:45.927196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c979fd0-f02b-412d-a679-060c40389edc None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:45.927481 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c979fd0-f02b-412d-a679-060c40389edc None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:45.927759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c979fd0-f02b-412d-a679-060c40389edc None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:45.928575 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c979fd0-f02b-412d-a679-060c40389edc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:45.928888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c979fd0-f02b-412d-a679-060c40389edc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:45.930437 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c979fd0-f02b-412d-a679-060c40389edc None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:45.944992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1f33d-6bee-4353-aa68-fec74d769e58 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:45.950321 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 373/731] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 3 12:40:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:45.951661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0629fb2-8035-4700-a2dc-5b7621e3dda4 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:45.951950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0629fb2-8035-4700-a2dc-5b7621e3dda4 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:45.952282 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0629fb2-8035-4700-a2dc-5b7621e3dda4 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:45.955588 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 374/732] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:45.956748 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb9322-a5fc-4804-9e5b-c789a8ec94c8 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:45.957032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb9322-a5fc-4804-9e5b-c789a8ec94c8 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:45.957310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb9322-a5fc-4804-9e5b-c789a8ec94c8 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:45.958758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 375/733] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:45.959345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb9322-a5fc-4804-9e5b-c789a8ec94c8 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:45.982630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98aced9-46d7-49dd-b26c-c18c60855d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:45.983294 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98aced9-46d7-49dd-b26c-c18c60855d35 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:45.983600 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98aced9-46d7-49dd-b26c-c18c60855d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:45.983918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98aced9-46d7-49dd-b26c-c18c60855d35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:45.994581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98aced9-46d7-49dd-b26c-c18c60855d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:45.996007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98aced9-46d7-49dd-b26c-c18c60855d35 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:46.010612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-b98aced9-46d7-49dd-b26c-c18c60855d35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:46.010612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:46.010612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:46.010612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:46.010612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:46.010612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:46.010612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:46.010612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:46.010612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:46.010612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:46.010612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:46.010612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:46.010612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:46.010612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:46.010612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:46.011607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:46.012509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:46.012509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:46.012509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:46.012509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:46.012509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 359/734] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:46.013220 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effebfc-a45e-481e-894e-49a897b745e7 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:46.013220 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effebfc-a45e-481e-894e-49a897b745e7 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:46.013374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effebfc-a45e-481e-894e-49a897b745e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:46.031822 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-5effebfc-a45e-481e-894e-49a897b745e7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:46.032926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-5effebfc-a45e-481e-894e-49a897b745e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:46.078025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb9322-a5fc-4804-9e5b-c789a8ec94c8 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:46.102696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 360/735] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:46.103926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e178b-7d03-4e9f-9ff7-a1d451258df7 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:46.104216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e178b-7d03-4e9f-9ff7-a1d451258df7 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:46.104494 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385e178b-7d03-4e9f-9ff7-a1d451258df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:46.106967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:46.117748 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1b805-ebbf-43dc-801b-8310dafd00ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:46.118364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1b805-ebbf-43dc-801b-8310dafd00ae None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:46.118674 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1b805-ebbf-43dc-801b-8310dafd00ae None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:46.118949 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1b805-ebbf-43dc-801b-8310dafd00ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:46.131754 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-385e178b-7d03-4e9f-9ff7-a1d451258df7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:46.132957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-385e178b-7d03-4e9f-9ff7-a1d451258df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:46.149402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:46.187533 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1b805-ebbf-43dc-801b-8310dafd00ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:46.188976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1b805-ebbf-43dc-801b-8310dafd00ae None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:46.199632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 361/736] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 03 12:40:46.203673 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e178b-7d03-4e9f-9ff7-a1d451258df7 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:46.288557 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 376/737] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 03 12:40:46.289663 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f11115-a8af-433f-af17-00f159410ae4 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:46.289954 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f11115-a8af-433f-af17-00f159410ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:46.290231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f11115-a8af-433f-af17-00f159410ae4 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:46.291757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 377/738] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:46.292505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f11115-a8af-433f-af17-00f159410ae4 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:46.303644 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385e178b-7d03-4e9f-9ff7-a1d451258df7 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:46.322639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:46.337452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99c23b-374f-4e36-a041-27cf1f03e80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:46.338067 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99c23b-374f-4e36-a041-27cf1f03e80d None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:46.338350 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99c23b-374f-4e36-a041-27cf1f03e80d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:46.338658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99c23b-374f-4e36-a041-27cf1f03e80d None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:46.339354 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99c23b-374f-4e36-a041-27cf1f03e80d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:46.339660 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99c23b-374f-4e36-a041-27cf1f03e80d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:46.341031 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99c23b-374f-4e36-a041-27cf1f03e80d None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:46.352429 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 362/739] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:46.353178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99c23b-374f-4e36-a041-27cf1f03e80d None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:46.368974 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:46.394759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f11115-a8af-433f-af17-00f159410ae4 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:46.428832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcaa23c-c8d4-4d28-ae1a-9542e3653f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:46.429443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcaa23c-c8d4-4d28-ae1a-9542e3653f25 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:46.429723 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcaa23c-c8d4-4d28-ae1a-9542e3653f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:46.429995 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcaa23c-c8d4-4d28-ae1a-9542e3653f25 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:46.430713 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcaa23c-c8d4-4d28-ae1a-9542e3653f25 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:46.432071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcaa23c-c8d4-4d28-ae1a-9542e3653f25 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:46.447295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 378/740] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 3 12:40:46 2022] POST /identity/v3/services => generated 240 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:40:46.448097 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcaa23c-c8d4-4d28-ae1a-9542e3653f25 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:46.454593 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99c23b-374f-4e36-a041-27cf1f03e80d None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:46.500165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ef470-fa67-4e65-ab0c-e8de5be76ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:46.501115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ef470-fa67-4e65-ab0c-e8de5be76ff7 None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:46.501567 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ef470-fa67-4e65-ab0c-e8de5be76ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:46.501840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ef470-fa67-4e65-ab0c-e8de5be76ff7 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:46.502592 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ef470-fa67-4e65-ab0c-e8de5be76ff7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:46.504040 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ef470-fa67-4e65-ab0c-e8de5be76ff7 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:46.522638 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 363/741] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 3 12:40:46 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:40:46.523746 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8f4be-f557-4170-b124-ab4d8ec7fc1a None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:46.524050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8f4be-f557-4170-b124-ab4d8ec7fc1a None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:46.524307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8f4be-f557-4170-b124-ab4d8ec7fc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:46.544762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-f6e8f4be-f557-4170-b124-ab4d8ec7fc1a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:46.545885 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f6e8f4be-f557-4170-b124-ab4d8ec7fc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:46.611748 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:46.623329 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcaa23c-c8d4-4d28-ae1a-9542e3653f25 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:46.624003 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:46.639261 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 364/742] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:40:46.645156 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8f4be-f557-4170-b124-ab4d8ec7fc1a None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:46.665336 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013dff5-09e0-4543-b284-5ef526d421dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:46.666004 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013dff5-09e0-4543-b284-5ef526d421dd None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:46.666289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013dff5-09e0-4543-b284-5ef526d421dd None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:46.667206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013dff5-09e0-4543-b284-5ef526d421dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:46.679291 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013dff5-09e0-4543-b284-5ef526d421dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:46.680776 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013dff5-09e0-4543-b284-5ef526d421dd None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:46.701802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-0013dff5-09e0-4543-b284-5ef526d421dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:46.701802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:46.701802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:46.701802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:46.701802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:46.701802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:46.701802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:46.701802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:46.701802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:46.701802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:46.701802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:46.701802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:46.701802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:46.701802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:46.701802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:46.702870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:46.703786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:46.703786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:46.703786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:46.703786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:46.703786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 379/743] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:46.704488 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013dff5-09e0-4543-b284-5ef526d421dd None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:46.764805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8f4be-f557-4170-b124-ab4d8ec7fc1a None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:46.814335 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9232006-860f-4045-8cb3-6a82bb65ac88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:46.815336 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9232006-860f-4045-8cb3-6a82bb65ac88 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:46.815690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9232006-860f-4045-8cb3-6a82bb65ac88 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:46.816469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9232006-860f-4045-8cb3-6a82bb65ac88 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:46.825091 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013dff5-09e0-4543-b284-5ef526d421dd None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:46.826670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9232006-860f-4045-8cb3-6a82bb65ac88 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:46.829611 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9232006-860f-4045-8cb3-6a82bb65ac88 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:46.846001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-f9232006-860f-4045-8cb3-6a82bb65ac88 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 12:40:46.846001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:46.846001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:46.846001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:46.846001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:46.846001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:46.846001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:46.846001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:46.846001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:46.846001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:46.846001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:46.846001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:46.846001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:46.846001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:46.846001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:46.846001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:46.847048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:46.848015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:46.848015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:46.848015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:46.848015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:46.848015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:46.848015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:46.848015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:46.848015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:46.848015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 12:40:46.848015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:46.848015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 365/744] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:40:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:46.849104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9232006-860f-4045-8cb3-6a82bb65ac88 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:46.874030 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e08c1-bb08-4ddc-8003-bec6f4508144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:46.874758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e08c1-bb08-4ddc-8003-bec6f4508144 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:46.875038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e08c1-bb08-4ddc-8003-bec6f4508144 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:46.878538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e08c1-bb08-4ddc-8003-bec6f4508144 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:46.895206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e08c1-bb08-4ddc-8003-bec6f4508144 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:46.901496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-588e08c1-bb08-4ddc-8003-bec6f4508144 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:46.901496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:46.901496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:46.901496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:46.901496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:46.901496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:46.901496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:46.901496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:46.901496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:46.901496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:46.901496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:46.901496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:46.901496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:46.901496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:46.901496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:46.901496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:46.902460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:46.902460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:46.902460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:46.902460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:46.902460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:46.902460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:46.902460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:46.902460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:46.902460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:46.902460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:46.902460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:46.902460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:46.902460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:46.902460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:46.902460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:46.902460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:46.903362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:46.903362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:46.903362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:46.903362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:46.903362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:46.903362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:46.903362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:46.903362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:46.903362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:46.903362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:46.903362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:46.903362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:46.904083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 380/745] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:40:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:46.909758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e08c1-bb08-4ddc-8003-bec6f4508144 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:46.910396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:46.922204 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:46.990164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9232006-860f-4045-8cb3-6a82bb65ac88 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:47.018656 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e08c1-bb08-4ddc-8003-bec6f4508144 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:47.035397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbceec-7fb1-42b9-b389-666cf54ab6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:47.036030 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbceec-7fb1-42b9-b389-666cf54ab6ba None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:47.036569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbceec-7fb1-42b9-b389-666cf54ab6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:47.036854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbceec-7fb1-42b9-b389-666cf54ab6ba None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:47.037961 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbceec-7fb1-42b9-b389-666cf54ab6ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:47.038279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbceec-7fb1-42b9-b389-666cf54ab6ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:47.039713 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbceec-7fb1-42b9-b389-666cf54ab6ba None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:47.055535 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e35902-176e-4faa-adaf-3d43858375fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:47.055902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 366/746] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:47.056161 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e35902-176e-4faa-adaf-3d43858375fe None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:47.056440 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e35902-176e-4faa-adaf-3d43858375fe None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:47.056554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbceec-7fb1-42b9-b389-666cf54ab6ba None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:47.061474 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e35902-176e-4faa-adaf-3d43858375fe None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:47.063096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e35902-176e-4faa-adaf-3d43858375fe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:47.063414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e35902-176e-4faa-adaf-3d43858375fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:47.064917 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e35902-176e-4faa-adaf-3d43858375fe None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:47.092200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 381/747] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 3 12:40:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:47.098235 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e35902-176e-4faa-adaf-3d43858375fe None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:47.154977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:47.182230 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbceec-7fb1-42b9-b389-666cf54ab6ba None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:47.198548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e35902-176e-4faa-adaf-3d43858375fe None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:47.199220 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:47.223390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fad6a-f4fb-46cf-82b1-1f78f262dc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:47.224397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fad6a-f4fb-46cf-82b1-1f78f262dc36 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:47.224902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fad6a-f4fb-46cf-82b1-1f78f262dc36 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:47.225401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fad6a-f4fb-46cf-82b1-1f78f262dc36 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:47.226536 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594fad6a-f4fb-46cf-82b1-1f78f262dc36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:47.227047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594fad6a-f4fb-46cf-82b1-1f78f262dc36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:47.228646 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594fad6a-f4fb-46cf-82b1-1f78f262dc36 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:47.234757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b262e8-cf1b-4285-b8a8-8861093f9b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:47.235274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b262e8-cf1b-4285-b8a8-8861093f9b49 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:47.235560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b262e8-cf1b-4285-b8a8-8861093f9b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:47.235834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b262e8-cf1b-4285-b8a8-8861093f9b49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:47.239374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 367/748] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:47.240505 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fad6a-f4fb-46cf-82b1-1f78f262dc36 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:47.243121 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b262e8-cf1b-4285-b8a8-8861093f9b49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:47.245249 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b262e8-cf1b-4285-b8a8-8861093f9b49 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:47.257542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-c0b262e8-cf1b-4285-b8a8-8861093f9b49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:47.257542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:47.257542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:47.257542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:47.257542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:47.257542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:47.257542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:47.257542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:47.257542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:47.257542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:47.257542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:47.257542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:47.257542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:47.257542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:47.257542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:47.258472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:47.259433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:47.259433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:47.259433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:47.259433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:47.259433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:47.259433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:47.259433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:47.259433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:47.264081 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 382/749] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:47.264988 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b262e8-cf1b-4285-b8a8-8861093f9b49 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:47.354668 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fad6a-f4fb-46cf-82b1-1f78f262dc36 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:47.368222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b262e8-cf1b-4285-b8a8-8861093f9b49 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:47.389301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:47.393793 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed03622-7cba-441b-9b79-a09f2297e555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:47.394714 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed03622-7cba-441b-9b79-a09f2297e555 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:47.395202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed03622-7cba-441b-9b79-a09f2297e555 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:47.395671 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed03622-7cba-441b-9b79-a09f2297e555 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:47.405224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d77ff3-1937-4d53-b467-3a1993151a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:47.405894 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d77ff3-1937-4d53-b467-3a1993151a17 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:47.406181 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d77ff3-1937-4d53-b467-3a1993151a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:47.406510 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d77ff3-1937-4d53-b467-3a1993151a17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:47.417223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d77ff3-1937-4d53-b467-3a1993151a17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:47.418690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d77ff3-1937-4d53-b467-3a1993151a17 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:47.433084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-03d77ff3-1937-4d53-b467-3a1993151a17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:47.433084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:47.433084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:47.433084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:47.433084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:47.433084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:47.433084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:47.433084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:47.433084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:47.433084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:47.433084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:47.433084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:47.433084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:47.433084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:47.433084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:47.433990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:47.435208 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:47.435208 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:47.435208 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:47.435208 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:47.435208 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 383/750] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:47.435928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d77ff3-1937-4d53-b467-3a1993151a17 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:47.449484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:47.475976 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed03622-7cba-441b-9b79-a09f2297e555 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:47.477783 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed03622-7cba-441b-9b79-a09f2297e555 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:47.540237 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d77ff3-1937-4d53-b467-3a1993151a17 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:47.577668 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213af0e-4382-47bb-a65c-d971d13fbe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:47.578304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213af0e-4382-47bb-a65c-d971d13fbe74 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:47.578618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213af0e-4382-47bb-a65c-d971d13fbe74 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:47.578897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213af0e-4382-47bb-a65c-d971d13fbe74 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:47.579654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213af0e-4382-47bb-a65c-d971d13fbe74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:47.579962 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213af0e-4382-47bb-a65c-d971d13fbe74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:47.581472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213af0e-4382-47bb-a65c-d971d13fbe74 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:47.592737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 368/751] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:40:47.594218 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 384/752] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:47.594627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed03622-7cba-441b-9b79-a09f2297e555 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:47.595078 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213af0e-4382-47bb-a65c-d971d13fbe74 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:47.632367 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:47.694886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:47.707091 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213af0e-4382-47bb-a65c-d971d13fbe74 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:47.708641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed03622-7cba-441b-9b79-a09f2297e555 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:47.744769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83592053-5ab1-43c7-8c93-79250712887b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:47.745422 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83592053-5ab1-43c7-8c93-79250712887b None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:47.745710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83592053-5ab1-43c7-8c93-79250712887b None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:47.745986 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83592053-5ab1-43c7-8c93-79250712887b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:47.746676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce7235-f6b6-4d8a-b3e6-f6648139b4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:47.747597 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce7235-f6b6-4d8a-b3e6-f6648139b4fc None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:47.748195 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce7235-f6b6-4d8a-b3e6-f6648139b4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:47.748669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce7235-f6b6-4d8a-b3e6-f6648139b4fc None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:47.750197 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce7235-f6b6-4d8a-b3e6-f6648139b4fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:47.750763 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce7235-f6b6-4d8a-b3e6-f6648139b4fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:47.752387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce7235-f6b6-4d8a-b3e6-f6648139b4fc None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:47.758449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83592053-5ab1-43c7-8c93-79250712887b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:47.760141 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83592053-5ab1-43c7-8c93-79250712887b None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:47.766749 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 369/753] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:47.768009 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce7235-f6b6-4d8a-b3e6-f6648139b4fc None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:47.777584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-83592053-5ab1-43c7-8c93-79250712887b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:47.777584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:47.777584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:47.777584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:47.777584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:47.777584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:47.777584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:47.777584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:47.777584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:47.777584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:47.777584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:47.777584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:47.777584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:47.777584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:47.777584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:47.778548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:47.779492 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:47.779492 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:47.779492 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:47.779492 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:47.783687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 385/754] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:47.784449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83592053-5ab1-43c7-8c93-79250712887b None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:47.871229 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:47.881597 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce7235-f6b6-4d8a-b3e6-f6648139b4fc None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:47.894554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83592053-5ab1-43c7-8c93-79250712887b None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:47.918330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d91e5-6cb0-4af0-872a-1d2f2ae91640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:47.919228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d91e5-6cb0-4af0-872a-1d2f2ae91640 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:47.919609 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d91e5-6cb0-4af0-872a-1d2f2ae91640 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:47.919991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d91e5-6cb0-4af0-872a-1d2f2ae91640 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:47.920871 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d91e5-6cb0-4af0-872a-1d2f2ae91640 None admin] RBAC: Adding query filter params (role.id=ac03c5aaca774e3196210f3bdb442418, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=3ae6eeb3d4f94925a5e1dd497d352d2e) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:47.921838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d91e5-6cb0-4af0-872a-1d2f2ae91640 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:47.923585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d91e5-6cb0-4af0-872a-1d2f2ae91640 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:47.934513 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 370/755] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:40:47 2022] GET /identity/v3/role_assignments?user.id=3ae6eeb3d4f94925a5e1dd497d352d2e&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=ac03c5aaca774e3196210f3bdb442418 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:47.934513 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:47.934513 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d91e5-6cb0-4af0-872a-1d2f2ae91640 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:47.937116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378ce38-7448-4353-8f33-da745b97a27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:47.937116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378ce38-7448-4353-8f33-da745b97a27e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:47.937116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378ce38-7448-4353-8f33-da745b97a27e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:47.937116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378ce38-7448-4353-8f33-da745b97a27e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:47.957750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378ce38-7448-4353-8f33-da745b97a27e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:47.959279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378ce38-7448-4353-8f33-da745b97a27e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:47.983794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-1378ce38-7448-4353-8f33-da745b97a27e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:40:47.983794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:47.983794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:47.983794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:47.983794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:47.983794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:47.983794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:47.983794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:47.983794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:47.983794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:47.983794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:47.983794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:47.983794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:47.983794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:47.983794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:47.983794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:47.985362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:47.986365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:47.986365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:47.986365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:47.986365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:47.986365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:47.986365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:47.986365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:47.986365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:47.986365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:40:47.986365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:47.986365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 386/756] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 3 12:40:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:47.994104 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378ce38-7448-4353-8f33-da745b97a27e None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:48.073433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d91e5-6cb0-4af0-872a-1d2f2ae91640 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:48.110630 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378ce38-7448-4353-8f33-da745b97a27e None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:48.112005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3a1a8-1feb-49cc-9ddb-637b5577c5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:48.112634 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3a1a8-1feb-49cc-9ddb-637b5577c5da None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:48.113044 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3a1a8-1feb-49cc-9ddb-637b5577c5da None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:48.113296 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3a1a8-1feb-49cc-9ddb-637b5577c5da None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:48.114012 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3a1a8-1feb-49cc-9ddb-637b5577c5da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:48.114351 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3a1a8-1feb-49cc-9ddb-637b5577c5da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:48.115880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3a1a8-1feb-49cc-9ddb-637b5577c5da None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:48.129501 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 371/757] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:48.135313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3a1a8-1feb-49cc-9ddb-637b5577c5da None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:48.151084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7c2eb-cb68-4734-ae80-78e79c26a13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:48.152149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7c2eb-cb68-4734-ae80-78e79c26a13e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:48.152657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7c2eb-cb68-4734-ae80-78e79c26a13e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:48.153135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7c2eb-cb68-4734-ae80-78e79c26a13e None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:48.154146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7c2eb-cb68-4734-ae80-78e79c26a13e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:48.154734 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7c2eb-cb68-4734-ae80-78e79c26a13e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:48.157033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be7c2eb-cb68-4734-ae80-78e79c26a13e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:48.178276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 387/758] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 3 12:40:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:48.184572 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7c2eb-cb68-4734-ae80-78e79c26a13e None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:48.188061 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:48.202896 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:48.249104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3a1a8-1feb-49cc-9ddb-637b5577c5da None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:48.288929 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660e6b6-7b46-4474-8f0b-838a89492a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:48.289581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660e6b6-7b46-4474-8f0b-838a89492a20 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:48.289955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660e6b6-7b46-4474-8f0b-838a89492a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:48.290130 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660e6b6-7b46-4474-8f0b-838a89492a20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:48.297608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660e6b6-7b46-4474-8f0b-838a89492a20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:48.299367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660e6b6-7b46-4474-8f0b-838a89492a20 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:48.302705 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7c2eb-cb68-4734-ae80-78e79c26a13e None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:48.311238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-e660e6b6-7b46-4474-8f0b-838a89492a20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:48.311238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:48.311238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:48.311238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:48.311238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:48.311238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:48.311238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:48.311238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:48.311238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:48.311238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:48.311238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:48.311238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:48.311238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:48.311238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:48.311238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:48.312234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:48.313151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:48.313151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:48.313151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:48.313151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:48.313151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:48.313151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:48.313151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:48.313151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:48.313151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 372/759] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:48.318395 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660e6b6-7b46-4474-8f0b-838a89492a20 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:48.344513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282de13e-b73e-45c1-9018-32978cd08c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:48.345642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282de13e-b73e-45c1-9018-32978cd08c8c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:48.346192 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282de13e-b73e-45c1-9018-32978cd08c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:48.346772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282de13e-b73e-45c1-9018-32978cd08c8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:48.358387 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282de13e-b73e-45c1-9018-32978cd08c8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:48.360200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282de13e-b73e-45c1-9018-32978cd08c8c None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:48.376879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-282de13e-b73e-45c1-9018-32978cd08c8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:48.376879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:48.376879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:48.376879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:48.376879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:48.376879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:48.376879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:48.376879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:48.376879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:48.376879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:48.376879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:48.376879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:48.376879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:48.376879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:48.376879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:48.377823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:48.378778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:48.378778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:48.378778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:48.378778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:48.381156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 388/760] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:48.387085 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282de13e-b73e-45c1-9018-32978cd08c8c None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:48.424503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660e6b6-7b46-4474-8f0b-838a89492a20 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:48.446016 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:48.449494 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:48.464275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4aa41b-8bc4-489e-9560-5127e8a4936f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:48.465051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4aa41b-8bc4-489e-9560-5127e8a4936f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:48.465264 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4aa41b-8bc4-489e-9560-5127e8a4936f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:48.465542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4aa41b-8bc4-489e-9560-5127e8a4936f None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:48.466302 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4aa41b-8bc4-489e-9560-5127e8a4936f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:48.466644 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4aa41b-8bc4-489e-9560-5127e8a4936f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:48.468712 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4aa41b-8bc4-489e-9560-5127e8a4936f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:48.482045 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 373/761] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:48.487951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4aa41b-8bc4-489e-9560-5127e8a4936f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:48.503599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282de13e-b73e-45c1-9018-32978cd08c8c None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:48.549340 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51e8a9-0655-4a4a-8e36-23ee4916dc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:48.550432 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51e8a9-0655-4a4a-8e36-23ee4916dc63 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:48.551084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51e8a9-0655-4a4a-8e36-23ee4916dc63 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:48.553896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b51e8a9-0655-4a4a-8e36-23ee4916dc63 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:48.553896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51e8a9-0655-4a4a-8e36-23ee4916dc63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:48.553896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51e8a9-0655-4a4a-8e36-23ee4916dc63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:48.554712 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b51e8a9-0655-4a4a-8e36-23ee4916dc63 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:48.569927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 389/762] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:48.576052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51e8a9-0655-4a4a-8e36-23ee4916dc63 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:48.595878 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4aa41b-8bc4-489e-9560-5127e8a4936f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:48.632412 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f072ab-2fd8-4285-b599-1cfe370593b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:48.633126 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f072ab-2fd8-4285-b599-1cfe370593b6 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:48.633410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f072ab-2fd8-4285-b599-1cfe370593b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:48.633683 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f072ab-2fd8-4285-b599-1cfe370593b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:48.634477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f072ab-2fd8-4285-b599-1cfe370593b6 None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=cea54cb772344612aec4686d484de89f) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:48.634810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f072ab-2fd8-4285-b599-1cfe370593b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:48.636240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f072ab-2fd8-4285-b599-1cfe370593b6 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:48.650515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 374/763] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:40:48 2022] GET /identity/v3/role_assignments?user.id=cea54cb772344612aec4686d484de89f&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:48.703650 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b51e8a9-0655-4a4a-8e36-23ee4916dc63 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:48.720893 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:48.744046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005af360-b14c-4343-b657-317acb6d7978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:48.744717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005af360-b14c-4343-b657-317acb6d7978 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:48.745001 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005af360-b14c-4343-b657-317acb6d7978 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:48.745278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005af360-b14c-4343-b657-317acb6d7978 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:48.753699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005af360-b14c-4343-b657-317acb6d7978 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:48.755204 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005af360-b14c-4343-b657-317acb6d7978 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:48.768390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-005af360-b14c-4343-b657-317acb6d7978 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:48.768390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:48.768390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:48.768390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:48.768390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:48.768390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:48.768390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:48.768390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:48.768390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:48.768390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:48.768390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:48.768390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:48.768390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:48.768390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:48.768390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:48.769366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:48.770315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:48.770315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:48.770315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:48.770315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:48.770315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:48.770315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:48.770315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:48.770315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:48.770315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 390/764] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:48.775908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f072ab-2fd8-4285-b599-1cfe370593b6 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:48.801137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b72315-642f-4a03-aec6-e6105ea38465 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:48.801503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b72315-642f-4a03-aec6-e6105ea38465 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:48.801709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b72315-642f-4a03-aec6-e6105ea38465 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:48.803399 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 391/765] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:48.811345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6d343-902e-4724-81e2-b51cb6d40d6f None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:48.811638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6d343-902e-4724-81e2-b51cb6d40d6f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:48.812450 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda6d343-902e-4724-81e2-b51cb6d40d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:48.832221 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-fda6d343-902e-4724-81e2-b51cb6d40d6f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:48.833452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-fda6d343-902e-4724-81e2-b51cb6d40d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:48.851452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:48.912402 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 392/766] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 12:40:48.918622 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff107c-6e0f-4151-ab66-998424bbe205 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:48.918923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff107c-6e0f-4151-ab66-998424bbe205 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:48.919274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff107c-6e0f-4151-ab66-998424bbe205 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:48.924444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 393/767] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:48.928637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f072ab-2fd8-4285-b599-1cfe370593b6 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:48.935873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1064c-84a0-4bd3-8314-70086e37b882 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:48.936177 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1064c-84a0-4bd3-8314-70086e37b882 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:48.936431 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1064c-84a0-4bd3-8314-70086e37b882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:48.959316 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-2bd1064c-84a0-4bd3-8314-70086e37b882 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:48.960521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-2bd1064c-84a0-4bd3-8314-70086e37b882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:48.968449 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b09c29-b5e5-4d94-9eb4-edf7ebe29b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:48.969374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b09c29-b5e5-4d94-9eb4-edf7ebe29b23 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:48.969890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b09c29-b5e5-4d94-9eb4-edf7ebe29b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:48.970380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b09c29-b5e5-4d94-9eb4-edf7ebe29b23 None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:48.971414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b09c29-b5e5-4d94-9eb4-edf7ebe29b23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:48.971952 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b09c29-b5e5-4d94-9eb4-edf7ebe29b23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:48.973596 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b09c29-b5e5-4d94-9eb4-edf7ebe29b23 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:48.986980 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 375/768] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:48.988236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b09c29-b5e5-4d94-9eb4-edf7ebe29b23 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:49.042505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 394/769] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:40:49.042505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1064c-84a0-4bd3-8314-70086e37b882 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:49.072010 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:49.118766 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b09c29-b5e5-4d94-9eb4-edf7ebe29b23 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:49.121476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:49.141368 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1064c-84a0-4bd3-8314-70086e37b882 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:49.157994 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ebe6d-b9bd-4b97-b0f9-1cc9406ef329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:49.158925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ebe6d-b9bd-4b97-b0f9-1cc9406ef329 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:49.159375 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ebe6d-b9bd-4b97-b0f9-1cc9406ef329 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:49.159865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ebe6d-b9bd-4b97-b0f9-1cc9406ef329 None admin] PATH_INFO: `/v3/services/network` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:49.160700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ebe6d-b9bd-4b97-b0f9-1cc9406ef329 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:49.162270 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ebe6d-b9bd-4b97-b0f9-1cc9406ef329 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-875ebe6d-b9bd-4b97-b0f9-1cc9406ef329 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:49.172224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:49.173199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:49.173199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:49.173199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:49.173199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:49.173199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:49.173199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:49.173199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:40:49.173199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:40:49.173199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:40:49.173199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:40:49.173199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:49.173199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:49.173199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:49.173199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:49.173199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:49.173199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:49.174043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:49.174043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:49.174043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:49.174043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:49.174043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:49.174043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:49.174043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:49.174043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:49.174043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:49.174043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:49.174043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:40:49.174043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:40:49.174043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 12:40:49.174043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:49.176820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 376/770] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:48 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:49.177856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ebe6d-b9bd-4b97-b0f9-1cc9406ef329 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:49.183288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19af31-9567-4fa8-a069-7ec47db3195f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:49.184081 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19af31-9567-4fa8-a069-7ec47db3195f None admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:49.184716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19af31-9567-4fa8-a069-7ec47db3195f None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:49.185089 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a19af31-9567-4fa8-a069-7ec47db3195f None admin] PATH_INFO: `/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/7b634f567dca42dc95df175a4f36c1d1/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:49.187822 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19af31-9567-4fa8-a069-7ec47db3195f None admin] RBAC: Authorizing `identity:create_grant(project_id=c0774035ad0f41c986771ef4a41c7d60, user_id=7b634f567dca42dc95df175a4f36c1d1, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:49.189396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a19af31-9567-4fa8-a069-7ec47db3195f None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:49.206666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 395/771] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:40:49 2022] PUT /identity/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/7b634f567dca42dc95df175a4f36c1d1/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:40:49.207699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19af31-9567-4fa8-a069-7ec47db3195f None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:49.376779 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ebe6d-b9bd-4b97-b0f9-1cc9406ef329 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:49.410861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:49.413958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a19af31-9567-4fa8-a069-7ec47db3195f None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:49.441134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe67a69-fe7f-4060-9f4a-6abd7ad08e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:49.441902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe67a69-fe7f-4060-9f4a-6abd7ad08e8c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:49.442261 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe67a69-fe7f-4060-9f4a-6abd7ad08e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:49.442755 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe67a69-fe7f-4060-9f4a-6abd7ad08e8c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:49.443481 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe67a69-fe7f-4060-9f4a-6abd7ad08e8c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:49.445701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe67a69-fe7f-4060-9f4a-6abd7ad08e8c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:49.459255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-cbe67a69-fe7f-4060-9f4a-6abd7ad08e8c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 12:40:49.459255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:49.459255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:40:49.459255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:40:49.459255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:40:49.459255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:40:49.459255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:40:49.459255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:40:49.459255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 12:40:49.459255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:49.459255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:40:49.459255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:49.459255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:49.459255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:49.459255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:49.459255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:49.460361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:49.460361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:49.460361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:49.460361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:49.460361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:49.460361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:49.460361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:49.460361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:40:49.460361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:40:49.460361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:40:49.460361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:40:49.460361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:49.460361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:49.460361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:49.460361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:49.460361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:49.466309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:49.466309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:49.466309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:49.466309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:49.466309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:49.466309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:49.466309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:49.466309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:49.466309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:49.466309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:49.466309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:49.466309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:40:49.466309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:40:49.466309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 12:40:49.466309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:49.466309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 377/772] 10.222.0.65 () {56 vars in 1129 bytes} [Wed Aug 3 12:40:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:49.469821 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe67a69-fe7f-4060-9f4a-6abd7ad08e8c None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:49.474613 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb646e-7af8-44ef-b390-0026e21ddbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:49.475252 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb646e-7af8-44ef-b390-0026e21ddbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:49.475535 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb646e-7af8-44ef-b390-0026e21ddbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:49.475807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb646e-7af8-44ef-b390-0026e21ddbc6 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:49.477438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb646e-7af8-44ef-b390-0026e21ddbc6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:49.477765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb646e-7af8-44ef-b390-0026e21ddbc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:49.479183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb646e-7af8-44ef-b390-0026e21ddbc6 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:49.489895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 396/773] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 3 12:40:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:49.495046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb646e-7af8-44ef-b390-0026e21ddbc6 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:49.527796 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:49.603798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe67a69-fe7f-4060-9f4a-6abd7ad08e8c None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:49.646908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f2621-2cb8-4084-aa69-2b17e8361162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:49.647862 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f2621-2cb8-4084-aa69-2b17e8361162 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:49.648368 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f2621-2cb8-4084-aa69-2b17e8361162 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:49.648844 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f2621-2cb8-4084-aa69-2b17e8361162 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:49.649729 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f2621-2cb8-4084-aa69-2b17e8361162 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:49.650236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f2621-2cb8-4084-aa69-2b17e8361162 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:49.651843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f2621-2cb8-4084-aa69-2b17e8361162 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:49.662081 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 378/774] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 3 12:40:49 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:49.663553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f2621-2cb8-4084-aa69-2b17e8361162 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:49.680007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb646e-7af8-44ef-b390-0026e21ddbc6 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:49.721546 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f116ab6-cbb4-4cad-a819-9bbc67c0c020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:49.722212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f116ab6-cbb4-4cad-a819-9bbc67c0c020 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:49.722517 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f116ab6-cbb4-4cad-a819-9bbc67c0c020 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:49.726095 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f116ab6-cbb4-4cad-a819-9bbc67c0c020 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:49.726095 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f116ab6-cbb4-4cad-a819-9bbc67c0c020 None admin] RBAC: Adding query filter params (type=network) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:49.726095 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f116ab6-cbb4-4cad-a819-9bbc67c0c020 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:49.726095 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f116ab6-cbb4-4cad-a819-9bbc67c0c020 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:49.739956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 397/775] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 3 12:40:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:49.745967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f116ab6-cbb4-4cad-a819-9bbc67c0c020 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:49.798841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:49.826836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:49.830805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f2621-2cb8-4084-aa69-2b17e8361162 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:49.906941 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e9add-3036-4986-b058-64bf99bad624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:49.907495 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e9add-3036-4986-b058-64bf99bad624 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:49.907773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e9add-3036-4986-b058-64bf99bad624 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:49.908101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e9add-3036-4986-b058-64bf99bad624 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:49.908802 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e9add-3036-4986-b058-64bf99bad624 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:49.909108 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e9add-3036-4986-b058-64bf99bad624 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:49.910502 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e9add-3036-4986-b058-64bf99bad624 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:49.921810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 379/776] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 3 12:40:49 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:49.926747 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e9add-3036-4986-b058-64bf99bad624 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:49.956697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f116ab6-cbb4-4cad-a819-9bbc67c0c020 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:50.026214 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcb646-8824-4f69-a40d-9fb340d95587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:50.026875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcb646-8824-4f69-a40d-9fb340d95587 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:50.027159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcb646-8824-4f69-a40d-9fb340d95587 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:50.027436 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcb646-8824-4f69-a40d-9fb340d95587 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:50.028205 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbcb646-8824-4f69-a40d-9fb340d95587 None admin] RBAC: Adding query filter params (interface=public, service_id=f3ed5c562d604e2782723ec23ed6bffa, region_id=RegionOne) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:50.028520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbcb646-8824-4f69-a40d-9fb340d95587 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:50.031057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbcb646-8824-4f69-a40d-9fb340d95587 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:50.044206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 398/777] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 3 12:40:49 2022] GET /identity/v3/endpoints?service_id=f3ed5c562d604e2782723ec23ed6bffa&interface=public®ion_id=RegionOne => generated 194 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:50.050369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcb646-8824-4f69-a40d-9fb340d95587 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:50.127148 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e9add-3036-4986-b058-64bf99bad624 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:50.163507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361eaf8-af96-4323-9fcb-65edd4cdbd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:50.164048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361eaf8-af96-4323-9fcb-65edd4cdbd90 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:50.164330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361eaf8-af96-4323-9fcb-65edd4cdbd90 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:50.164605 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361eaf8-af96-4323-9fcb-65edd4cdbd90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:50.166519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361eaf8-af96-4323-9fcb-65edd4cdbd90 None admin] RBAC: Adding query filter params (interface=public, service_id=9f5cf532e7d64c5693c003bc07caaafb, region_id=RegionOne) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:50.166519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361eaf8-af96-4323-9fcb-65edd4cdbd90 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:50.167057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361eaf8-af96-4323-9fcb-65edd4cdbd90 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:50.175635 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 380/778] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 3 12:40:49 2022] GET /identity/v3/endpoints?service_id=9f5cf532e7d64c5693c003bc07caaafb&interface=public®ion_id=RegionOne => generated 194 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:50.181417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361eaf8-af96-4323-9fcb-65edd4cdbd90 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:50.218860 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:50.234574 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:50.244356 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcb646-8824-4f69-a40d-9fb340d95587 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:50.337403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361eaf8-af96-4323-9fcb-65edd4cdbd90 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:50.339185 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cc86b-93dc-499d-baa5-f795499d4f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:50.339826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cc86b-93dc-499d-baa5-f795499d4f81 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:50.340106 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cc86b-93dc-499d-baa5-f795499d4f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:50.340374 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cc86b-93dc-499d-baa5-f795499d4f81 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:50.341032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756cc86b-93dc-499d-baa5-f795499d4f81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:50.342398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756cc86b-93dc-499d-baa5-f795499d4f81 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:50.354925 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 399/779] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 3 12:40:50 2022] GET /identity/v3/services => generated 772 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:50.386126 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa904c0-eb9d-4863-b52f-0cead5d227b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:50.386926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa904c0-eb9d-4863-b52f-0cead5d227b4 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:50.387249 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa904c0-eb9d-4863-b52f-0cead5d227b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:50.387527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa904c0-eb9d-4863-b52f-0cead5d227b4 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:50.388196 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa904c0-eb9d-4863-b52f-0cead5d227b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:50.389569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa904c0-eb9d-4863-b52f-0cead5d227b4 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:50.400067 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 381/780] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 3 12:40:50 2022] GET /identity/v3/services => generated 772 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:50.465954 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e2a32-0f0d-4ef4-a6dc-e7ffb39735ee None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:50.466289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e2a32-0f0d-4ef4-a6dc-e7ffb39735ee None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:50.466524 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936e2a32-0f0d-4ef4-a6dc-e7ffb39735ee None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:50.468799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 400/781] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:50.474390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a856b04-ca59-4399-ac09-5529017b8e98 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:50.474723 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a856b04-ca59-4399-ac09-5529017b8e98 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:50.474975 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a856b04-ca59-4399-ac09-5529017b8e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:50.502241 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-9a856b04-ca59-4399-ac09-5529017b8e98 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:50.503535 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-9a856b04-ca59-4399-ac09-5529017b8e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:50.596039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 382/782] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:40:50.618231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e2a32-0f0d-4ef4-a6dc-e7ffb39735ee None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:50.733491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936e2a32-0f0d-4ef4-a6dc-e7ffb39735ee None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:50.768363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f22c0f-9c43-4102-86f6-0d719582394e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:50.769006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f22c0f-9c43-4102-86f6-0d719582394e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:50.769579 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f22c0f-9c43-4102-86f6-0d719582394e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:50.769852 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f22c0f-9c43-4102-86f6-0d719582394e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:50.781212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f22c0f-9c43-4102-86f6-0d719582394e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:50.786180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-75f22c0f-9c43-4102-86f6-0d719582394e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:50.786180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:50.786180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:50.786180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:50.786180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:50.786180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:50.786180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:50.786180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:50.786180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:50.786180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:50.786180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:50.786180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:50.786180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:50.786180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:50.786180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:50.786180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:50.787167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:50.787167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:50.787167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:50.787167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:50.787167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:50.787167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:50.787167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:50.787167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:50.787167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:50.787167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:50.787167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:50.787167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:50.787167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:50.787167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:50.787167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:50.787167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:50.788025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:50.788025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:50.788025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:50.788025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:50.788025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:50.788025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:50.788025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:50.788025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:50.788025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:50.788025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:50.788025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:50.788025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:50.788025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 401/783] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:50.794130 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a856b04-ca59-4399-ac09-5529017b8e98 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:50.823647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:50.916190 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7229c9c-ee9c-4334-b30f-2cc32ba640b5 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:50.916721 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7229c9c-ee9c-4334-b30f-2cc32ba640b5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:50.917138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7229c9c-ee9c-4334-b30f-2cc32ba640b5 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:50.918644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 402/784] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:40:50.923739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d05cef-94b3-4f7f-8d87-5c078d434bf9 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:50.924280 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d05cef-94b3-4f7f-8d87-5c078d434bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:50.924728 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d05cef-94b3-4f7f-8d87-5c078d434bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:50.931054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a856b04-ca59-4399-ac09-5529017b8e98 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:50.959228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-c3d05cef-94b3-4f7f-8d87-5c078d434bf9 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:50.960653 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c3d05cef-94b3-4f7f-8d87-5c078d434bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:50.973296 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:50.976976 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50379813-4bb1-40ba-b1d8-f936eb8df61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:50.977615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50379813-4bb1-40ba-b1d8-f936eb8df61d None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:50.977897 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50379813-4bb1-40ba-b1d8-f936eb8df61d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:50.978755 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50379813-4bb1-40ba-b1d8-f936eb8df61d None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:50.980495 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50379813-4bb1-40ba-b1d8-f936eb8df61d None admin] RBAC: Adding query filter params (name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:50.980899 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50379813-4bb1-40ba-b1d8-f936eb8df61d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:50.982289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50379813-4bb1-40ba-b1d8-f936eb8df61d None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:51.001169 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 383/785] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:51.004315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50379813-4bb1-40ba-b1d8-f936eb8df61d None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:51.089107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 403/786] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:40:51.126465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d05cef-94b3-4f7f-8d87-5c078d434bf9 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:51.142801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:51.179763 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50379813-4bb1-40ba-b1d8-f936eb8df61d None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:51.255250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec364ea-af93-48b8-86f7-c3fa4b679bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:51.256148 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec364ea-af93-48b8-86f7-c3fa4b679bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:51.256586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec364ea-af93-48b8-86f7-c3fa4b679bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:51.256995 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec364ea-af93-48b8-86f7-c3fa4b679bc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:51.279899 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec364ea-af93-48b8-86f7-c3fa4b679bc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:51.281847 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec364ea-af93-48b8-86f7-c3fa4b679bc0 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:51.308263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-6ec364ea-af93-48b8-86f7-c3fa4b679bc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:51.308263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:51.308263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:51.308263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:51.308263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:51.308263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:51.308263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:51.308263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:51.308263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:51.308263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:51.308263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:51.308263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:51.308263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:51.308263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:51.308263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:51.309633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:51.310737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:51.310737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:51.310737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:51.310737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:51.320478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 384/787] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:51.323473 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d05cef-94b3-4f7f-8d87-5c078d434bf9 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:51.326036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec364ea-af93-48b8-86f7-c3fa4b679bc0 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:51.398130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26873ed1-d996-489b-adad-8dc5beab54f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:51.399197 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26873ed1-d996-489b-adad-8dc5beab54f0 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:51.399686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26873ed1-d996-489b-adad-8dc5beab54f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:51.400188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26873ed1-d996-489b-adad-8dc5beab54f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:51.430221 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26873ed1-d996-489b-adad-8dc5beab54f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:51.435690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-26873ed1-d996-489b-adad-8dc5beab54f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:51.435690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:51.435690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:51.435690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:51.435690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:51.435690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:51.435690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:51.435690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:51.435690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:51.435690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:51.435690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:51.435690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:51.435690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:51.435690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:51.435690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:51.435690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:51.437371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:51.437371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:51.437371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:51.437371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:51.437371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:51.437371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:51.437371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:51.437371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:51.437371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:51.437371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:51.437371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:51.437371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:51.437371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:51.437371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:51.437371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:51.437371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:51.438419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:51.438419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:51.438419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:51.438419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:51.438419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:51.438419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:51.438419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:51.438419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:51.438419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:51.438419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:51.438419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:51.438419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:51.449832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 404/788] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:40:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:51.449978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:51.456259 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26873ed1-d996-489b-adad-8dc5beab54f0 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:51.500375 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec364ea-af93-48b8-86f7-c3fa4b679bc0 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:51.562716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:51.570192 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b05cf-b2f0-43cd-8ed4-be38eab47664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:51.571941 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b05cf-b2f0-43cd-8ed4-be38eab47664 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:51.572223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b05cf-b2f0-43cd-8ed4-be38eab47664 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:51.572493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b05cf-b2f0-43cd-8ed4-be38eab47664 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:51.573184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b05cf-b2f0-43cd-8ed4-be38eab47664 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:51.573484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b05cf-b2f0-43cd-8ed4-be38eab47664 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:51.574874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b05cf-b2f0-43cd-8ed4-be38eab47664 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:51.587054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 385/789] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:51.588250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fd36a-dd1c-4555-9e21-6696a443aa20 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:51.588530 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fd36a-dd1c-4555-9e21-6696a443aa20 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:51.588798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fd36a-dd1c-4555-9e21-6696a443aa20 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:51.590522 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 386/790] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:51.592272 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fd36a-dd1c-4555-9e21-6696a443aa20 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:51.621731 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26873ed1-d996-489b-adad-8dc5beab54f0 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:51.659266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73434938-f271-450a-804e-755a186fff57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:51.660031 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73434938-f271-450a-804e-755a186fff57 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:51.660496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73434938-f271-450a-804e-755a186fff57 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:51.660874 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73434938-f271-450a-804e-755a186fff57 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:51.661717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73434938-f271-450a-804e-755a186fff57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:51.662220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73434938-f271-450a-804e-755a186fff57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:51.663889 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73434938-f271-450a-804e-755a186fff57 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:51.673723 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 405/791] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:51.675224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208e6e3-eb1d-444e-a556-ea73342c7fc3 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:51.675651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208e6e3-eb1d-444e-a556-ea73342c7fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:51.676029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208e6e3-eb1d-444e-a556-ea73342c7fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:51.694703 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-d208e6e3-eb1d-444e-a556-ea73342c7fc3 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:51.696288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d208e6e3-eb1d-444e-a556-ea73342c7fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:51.705824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fd36a-dd1c-4555-9e21-6696a443aa20 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:51.741955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc191ad7-78b8-4e91-b5f7-ee314cdae5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:51.742894 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc191ad7-78b8-4e91-b5f7-ee314cdae5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:51.743323 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc191ad7-78b8-4e91-b5f7-ee314cdae5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:51.743839 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc191ad7-78b8-4e91-b5f7-ee314cdae5e2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:51.752117 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc191ad7-78b8-4e91-b5f7-ee314cdae5e2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:51.753854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc191ad7-78b8-4e91-b5f7-ee314cdae5e2 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:51.759512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:51.768876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-bc191ad7-78b8-4e91-b5f7-ee314cdae5e2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 12:40:51.768876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:51.768876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:51.768876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:51.768876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:51.768876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:51.768876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:51.768876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:51.768876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:51.768876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:51.768876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:51.768876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:51.768876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:51.768876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:51.768876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:51.768876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:51.770054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:51.781864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:51.781864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:51.781864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:51.781864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:51.781864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:51.781864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:51.781864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:51.781864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:51.781864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 12:40:51.781864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:51.783234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 406/792] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:40:51.783234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208e6e3-eb1d-444e-a556-ea73342c7fc3 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:51.783884 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 387/793] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 3 12:40:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:40:51.787540 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc191ad7-78b8-4e91-b5f7-ee314cdae5e2 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:51.851016 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:51.908237 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208e6e3-eb1d-444e-a556-ea73342c7fc3 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:51.916832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc191ad7-78b8-4e91-b5f7-ee314cdae5e2 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:51.944722 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3f30a-5d8c-4a8d-8b0c-8ce068ad3fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:51.945372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3f30a-5d8c-4a8d-8b0c-8ce068ad3fea None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:51.945656 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3f30a-5d8c-4a8d-8b0c-8ce068ad3fea None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:51.945923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3f30a-5d8c-4a8d-8b0c-8ce068ad3fea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:51.956593 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3f30a-5d8c-4a8d-8b0c-8ce068ad3fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:51.958026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3f30a-5d8c-4a8d-8b0c-8ce068ad3fea None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:51.959999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad8e53-4e48-40a9-9984-8eab7c963309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:51.960948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad8e53-4e48-40a9-9984-8eab7c963309 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:51.961429 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad8e53-4e48-40a9-9984-8eab7c963309 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:51.961899 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad8e53-4e48-40a9-9984-8eab7c963309 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:51.962930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad8e53-4e48-40a9-9984-8eab7c963309 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:51.963439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad8e53-4e48-40a9-9984-8eab7c963309 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:51.965226 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad8e53-4e48-40a9-9984-8eab7c963309 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:51.973397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-fbc3f30a-5d8c-4a8d-8b0c-8ce068ad3fea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:51.973397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:51.973397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:51.973397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:51.973397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:51.973397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:51.973397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:51.973397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:51.973397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:51.973397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:51.973397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:51.973397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:51.973397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:51.973397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:51.973397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:51.974292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:51.975220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:51.975220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:51.975220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:51.975220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:51.975220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 407/794] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:51.976790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3f30a-5d8c-4a8d-8b0c-8ce068ad3fea None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:51.988708 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 388/795] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 3 12:40:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:51.989809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad8e53-4e48-40a9-9984-8eab7c963309 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:52.027294 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:52.085293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3f30a-5d8c-4a8d-8b0c-8ce068ad3fea None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:52.121498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad8e53-4e48-40a9-9984-8eab7c963309 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:52.132659 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e99ea-9f2e-4a22-8563-608c4e820790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:52.133901 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e99ea-9f2e-4a22-8563-608c4e820790 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:52.134418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e99ea-9f2e-4a22-8563-608c4e820790 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:52.134550 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:52.135055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e99ea-9f2e-4a22-8563-608c4e820790 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:52.149064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e99ea-9f2e-4a22-8563-608c4e820790 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:52.157201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-7e0e99ea-9f2e-4a22-8563-608c4e820790 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:52.157201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:52.157201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:52.157201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:52.157201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:52.157201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:52.157201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:52.157201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:52.157201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:52.157201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:52.157201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:52.157201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:52.157201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:52.157201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:52.157201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:52.157201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:52.158145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:52.158145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:52.158145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:52.158145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:52.158145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:52.158145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:52.158145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:52.158145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:52.158145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:52.158145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:52.158145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:52.158145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:52.158145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:52.158145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:52.158145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:52.158145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:52.159041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:52.159041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:52.159041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:52.159041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:52.159041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:52.159041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:52.159041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:52.159041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:52.159041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:52.159041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:52.159041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:52.159041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:52.161237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb94725-7ed1-4b71-b1e3-0b43300020a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:52.161518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb94725-7ed1-4b71-b1e3-0b43300020a8 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:52.161802 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb94725-7ed1-4b71-b1e3-0b43300020a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:52.162075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb94725-7ed1-4b71-b1e3-0b43300020a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:52.163007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb94725-7ed1-4b71-b1e3-0b43300020a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:52.163310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb94725-7ed1-4b71-b1e3-0b43300020a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:52.164110 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 408/796] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:40:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:52.164670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb94725-7ed1-4b71-b1e3-0b43300020a8 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:52.165394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e99ea-9f2e-4a22-8563-608c4e820790 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:52.184648 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 389/797] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:52.185665 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb94725-7ed1-4b71-b1e3-0b43300020a8 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:52.299189 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e99ea-9f2e-4a22-8563-608c4e820790 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:52.302262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:52.317047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb94725-7ed1-4b71-b1e3-0b43300020a8 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:52.336653 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839fba3-2395-4806-baf9-07570020c78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:52.337303 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839fba3-2395-4806-baf9-07570020c78b None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:52.337585 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839fba3-2395-4806-baf9-07570020c78b None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:52.337859 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839fba3-2395-4806-baf9-07570020c78b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:52.348348 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839fba3-2395-4806-baf9-07570020c78b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:52.349792 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839fba3-2395-4806-baf9-07570020c78b None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:52.356364 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb6137-3694-40e2-8119-ee25b6e5e77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:52.357376 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb6137-3694-40e2-8119-ee25b6e5e77a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:52.357866 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb6137-3694-40e2-8119-ee25b6e5e77a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:52.358426 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb6137-3694-40e2-8119-ee25b6e5e77a None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:52.359911 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb6137-3694-40e2-8119-ee25b6e5e77a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:52.360438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb6137-3694-40e2-8119-ee25b6e5e77a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:52.362057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fb6137-3694-40e2-8119-ee25b6e5e77a None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:52.365231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-3839fba3-2395-4806-baf9-07570020c78b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:52.365231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:52.365231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:52.365231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:52.365231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:52.365231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:52.365231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:52.365231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:52.365231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:52.365231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:52.365231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:52.365231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:52.365231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:52.365231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:52.365231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:52.366201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:52.367149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:52.367149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:52.367149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:52.367149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:52.367149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 409/798] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:52.368800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839fba3-2395-4806-baf9-07570020c78b None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:52.376360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 390/799] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:52.377662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb6137-3694-40e2-8119-ee25b6e5e77a None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:52.412184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:52.475464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839fba3-2395-4806-baf9-07570020c78b None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:52.492048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fb6137-3694-40e2-8119-ee25b6e5e77a None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:52.510863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394797f6-5579-4432-92f9-86b3c31f1887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:52.511509 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394797f6-5579-4432-92f9-86b3c31f1887 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:52.511791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394797f6-5579-4432-92f9-86b3c31f1887 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:52.512062 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394797f6-5579-4432-92f9-86b3c31f1887 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:52.519268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394797f6-5579-4432-92f9-86b3c31f1887 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:52.520699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394797f6-5579-4432-92f9-86b3c31f1887 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:52.531252 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af8503-aaed-4d1c-990a-b07299ebcd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:52.532145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af8503-aaed-4d1c-990a-b07299ebcd17 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:52.532499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-394797f6-5579-4432-92f9-86b3c31f1887 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 12:40:52.532499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:52.532499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:52.532499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:52.532499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:52.532499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:52.532499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:52.532499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:52.532499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:52.532499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:52.532499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:52.532499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:52.532499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:52.532499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:52.532499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:52.532499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:52.533435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:52.534412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:52.534412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:52.534412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:52.534412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:52.534412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:52.534412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:52.534412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:52.534412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:52.534412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 12:40:52.534412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:52.534412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 410/800] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:40:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:52.535046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394797f6-5579-4432-92f9-86b3c31f1887 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:52.536922 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af8503-aaed-4d1c-990a-b07299ebcd17 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:52.537481 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af8503-aaed-4d1c-990a-b07299ebcd17 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:52.538589 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af8503-aaed-4d1c-990a-b07299ebcd17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:52.539176 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af8503-aaed-4d1c-990a-b07299ebcd17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:52.540809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42af8503-aaed-4d1c-990a-b07299ebcd17 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:52.553595 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:52.556889 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 391/801] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:52.558109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af8503-aaed-4d1c-990a-b07299ebcd17 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:52.647071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394797f6-5579-4432-92f9-86b3c31f1887 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:52.675702 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:52.686141 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42af8503-aaed-4d1c-990a-b07299ebcd17 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:52.697402 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360aee1b-073e-4774-b245-bbb35dc4253d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:52.698842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360aee1b-073e-4774-b245-bbb35dc4253d None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:52.699612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360aee1b-073e-4774-b245-bbb35dc4253d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:52.700178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360aee1b-073e-4774-b245-bbb35dc4253d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:52.713822 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360aee1b-073e-4774-b245-bbb35dc4253d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:52.715296 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360aee1b-073e-4774-b245-bbb35dc4253d None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:52.722518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf3075-a68f-4b87-a150-abdb19cf6094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:52.723145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf3075-a68f-4b87-a150-abdb19cf6094 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:52.723432 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf3075-a68f-4b87-a150-abdb19cf6094 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:52.724096 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf3075-a68f-4b87-a150-abdb19cf6094 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:52.725166 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf3075-a68f-4b87-a150-abdb19cf6094 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:52.725477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf3075-a68f-4b87-a150-abdb19cf6094 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:52.726918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf3075-a68f-4b87-a150-abdb19cf6094 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:52.729920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-360aee1b-073e-4774-b245-bbb35dc4253d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:52.729920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:52.729920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:52.729920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:52.729920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:52.729920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:52.729920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:52.729920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:52.729920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:52.729920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:52.729920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:52.729920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:52.729920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:52.729920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:52.729920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:52.730863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:52.731779 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:52.731779 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:52.731779 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:52.731779 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:52.731779 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 411/802] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:52.732068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360aee1b-073e-4774-b245-bbb35dc4253d None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:52.752868 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 392/803] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 3 12:40:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:52.754342 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c6ef2-1fb8-45a5-9c13-3b706b9bcc61 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:52.754811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c6ef2-1fb8-45a5-9c13-3b706b9bcc61 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:52.755728 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c6ef2-1fb8-45a5-9c13-3b706b9bcc61 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:52.757423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 393/804] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:52.758757 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c6ef2-1fb8-45a5-9c13-3b706b9bcc61 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:52.803066 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:52.837452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360aee1b-073e-4774-b245-bbb35dc4253d None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:52.869261 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c6ef2-1fb8-45a5-9c13-3b706b9bcc61 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:52.873399 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cf7be-f88c-4696-abab-790015d6db47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:52.874042 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cf7be-f88c-4696-abab-790015d6db47 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:52.874357 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cf7be-f88c-4696-abab-790015d6db47 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:52.874661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cf7be-f88c-4696-abab-790015d6db47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:52.881648 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886cf7be-f88c-4696-abab-790015d6db47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:52.883209 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886cf7be-f88c-4696-abab-790015d6db47 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:52.894573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-886cf7be-f88c-4696-abab-790015d6db47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:52.894573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:52.894573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:52.894573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:52.894573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:52.894573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:52.894573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:52.894573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:52.894573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:52.894573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:52.894573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:52.894573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:52.894573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:52.894573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:52.894573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:52.895644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:52.896739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:52.896739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:52.896739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:52.896739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:52.896739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:52.896739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:52.896739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:52.896739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:52.900317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 412/805] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:52.901050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cf7be-f88c-4696-abab-790015d6db47 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:52.906614 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e679c69-5da0-404f-a318-d91965f5d7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:52.907233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e679c69-5da0-404f-a318-d91965f5d7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:52.907515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e679c69-5da0-404f-a318-d91965f5d7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:52.907789 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e679c69-5da0-404f-a318-d91965f5d7b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:52.908488 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e679c69-5da0-404f-a318-d91965f5d7b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:52.908832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e679c69-5da0-404f-a318-d91965f5d7b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:52.910196 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e679c69-5da0-404f-a318-d91965f5d7b2 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:52.921806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 394/806] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:52.923014 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b81cb0-6d61-40d3-ace7-c9bb490abaa0 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:52.923301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b81cb0-6d61-40d3-ace7-c9bb490abaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:52.923580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b81cb0-6d61-40d3-ace7-c9bb490abaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:52.941320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-b5b81cb0-6d61-40d3-ace7-c9bb490abaa0 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:52.942542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-b5b81cb0-6d61-40d3-ace7-c9bb490abaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:52.945556 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:53.001282 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cf7be-f88c-4696-abab-790015d6db47 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:53.009110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 395/807] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:40:53.010161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6e5dd-27a4-4e0b-97d1-6d93dd6b651c None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:53.010446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6e5dd-27a4-4e0b-97d1-6d93dd6b651c None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:53.010751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6e5dd-27a4-4e0b-97d1-6d93dd6b651c None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:53.012110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 396/808] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:53.012676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6e5dd-27a4-4e0b-97d1-6d93dd6b651c None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:53.031751 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:53.040382 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1073703-0ccc-4543-a3f3-e9f715e478ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:53.041028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1073703-0ccc-4543-a3f3-e9f715e478ac None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:53.041308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1073703-0ccc-4543-a3f3-e9f715e478ac None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:53.041590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1073703-0ccc-4543-a3f3-e9f715e478ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:53.052611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1073703-0ccc-4543-a3f3-e9f715e478ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:53.054038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1073703-0ccc-4543-a3f3-e9f715e478ac None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:53.068180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-c1073703-0ccc-4543-a3f3-e9f715e478ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:53.068180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:53.068180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:53.068180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:53.068180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:53.068180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:53.068180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:53.068180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:53.068180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:53.068180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:53.068180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:53.068180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:53.068180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:53.068180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:53.068180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:53.069103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:53.070005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:53.070005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:53.070005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:53.070005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:53.070005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 413/809] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:53.070511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1073703-0ccc-4543-a3f3-e9f715e478ac None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:53.118961 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6e5dd-27a4-4e0b-97d1-6d93dd6b651c None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:53.151867 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b8ae3-f5d1-42fb-a797-a7f4df605d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:53.152415 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b8ae3-f5d1-42fb-a797-a7f4df605d53 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:53.152700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b8ae3-f5d1-42fb-a797-a7f4df605d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:53.152975 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b8ae3-f5d1-42fb-a797-a7f4df605d53 None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:53.153719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b8ae3-f5d1-42fb-a797-a7f4df605d53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:53.154023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b8ae3-f5d1-42fb-a797-a7f4df605d53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:53.155971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b8ae3-f5d1-42fb-a797-a7f4df605d53 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:53.164474 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:53.173288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1073703-0ccc-4543-a3f3-e9f715e478ac None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:53.173460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 397/810] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:53.173460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e7787-8cbf-429c-a221-67e55523acb4 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:53.173460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e7787-8cbf-429c-a221-67e55523acb4 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:53.173460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e7787-8cbf-429c-a221-67e55523acb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:53.190561 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-af1e7787-8cbf-429c-a221-67e55523acb4 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:53.191758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-af1e7787-8cbf-429c-a221-67e55523acb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:53.201199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491af3d-d31e-46de-89bd-f1f0e01fdf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:53.201829 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491af3d-d31e-46de-89bd-f1f0e01fdf40 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:53.202112 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491af3d-d31e-46de-89bd-f1f0e01fdf40 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:53.202909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491af3d-d31e-46de-89bd-f1f0e01fdf40 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:53.210177 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491af3d-d31e-46de-89bd-f1f0e01fdf40 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:53.211706 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491af3d-d31e-46de-89bd-f1f0e01fdf40 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:53.223455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-f491af3d-d31e-46de-89bd-f1f0e01fdf40 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:40:53.223455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:53.223455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:53.223455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:53.223455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:53.223455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:53.223455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:53.223455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:53.223455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:53.223455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:53.223455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:53.223455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:53.223455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:53.223455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:53.223455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:53.223455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:53.224424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:53.225448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:53.225448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:53.225448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:53.225448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:53.225448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:53.225448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:53.225448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:53.225448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:53.225448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:40:53.225448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:53.225448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 414/811] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 3 12:40:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:53.226052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491af3d-d31e-46de-89bd-f1f0e01fdf40 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:53.260586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 398/812] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:40:53.261411 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e7787-8cbf-429c-a221-67e55523acb4 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:53.266522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:53.326823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491af3d-d31e-46de-89bd-f1f0e01fdf40 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:53.360237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e7787-8cbf-429c-a221-67e55523acb4 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:53.360595 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2517dad-bfc0-48ac-b9ad-baeea7c8a41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:53.361233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2517dad-bfc0-48ac-b9ad-baeea7c8a41a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:53.361522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2517dad-bfc0-48ac-b9ad-baeea7c8a41a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:53.361794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2517dad-bfc0-48ac-b9ad-baeea7c8a41a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:53.363079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2517dad-bfc0-48ac-b9ad-baeea7c8a41a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:53.363802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2517dad-bfc0-48ac-b9ad-baeea7c8a41a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:53.372888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-a2517dad-bfc0-48ac-b9ad-baeea7c8a41a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 12:40:53.372888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:53.372888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:40:53.372888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:40:53.372888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:40:53.372888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:40:53.372888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:40:53.372888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:40:53.372888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 12:40:53.372888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:53.372888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:40:53.372888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:53.372888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:53.372888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:53.372888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:53.372888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:53.373946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:53.373946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:53.373946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:53.373946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:53.373946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:53.373946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:53.373946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:53.373946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:40:53.373946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:40:53.373946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:40:53.373946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:40:53.373946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:53.373946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:53.373946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:53.373946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:53.373946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:53.374841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:53.374841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:53.374841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:53.374841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:53.374841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:53.374841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:53.374841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:53.374841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:53.374841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:53.374841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:53.374841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:53.374841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:40:53.374841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:40:53.374841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 03 12:40:53.374841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:53.374841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 415/813] 10.222.0.65 () {56 vars in 1129 bytes} [Wed Aug 3 12:40:53 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:53.375705 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2517dad-bfc0-48ac-b9ad-baeea7c8a41a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:53.389376 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:53.396287 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489f2dc-ac53-418f-9ca5-d10628fddc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:53.396901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489f2dc-ac53-418f-9ca5-d10628fddc4e None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:53.397179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489f2dc-ac53-418f-9ca5-d10628fddc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:53.397451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e489f2dc-ac53-418f-9ca5-d10628fddc4e None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:53.398148 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489f2dc-ac53-418f-9ca5-d10628fddc4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:53.398449 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489f2dc-ac53-418f-9ca5-d10628fddc4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:53.399843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e489f2dc-ac53-418f-9ca5-d10628fddc4e None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:53.411335 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 399/814] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:53.412057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489f2dc-ac53-418f-9ca5-d10628fddc4e None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:53.477374 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2517dad-bfc0-48ac-b9ad-baeea7c8a41a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:53.479624 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:53.510756 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93460cc-024f-4964-9b16-c3e50050f067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:53.511015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e489f2dc-ac53-418f-9ca5-d10628fddc4e None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:53.511416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93460cc-024f-4964-9b16-c3e50050f067 None admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:53.511696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93460cc-024f-4964-9b16-c3e50050f067 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:53.511972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93460cc-024f-4964-9b16-c3e50050f067 None admin] PATH_INFO: `/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/3ae6eeb3d4f94925a5e1dd497d352d2e/roles/ac03c5aaca774e3196210f3bdb442418` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:53.526309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93460cc-024f-4964-9b16-c3e50050f067 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0774035ad0f41c986771ef4a41c7d60, user_id=3ae6eeb3d4f94925a5e1dd497d352d2e, role_id=ac03c5aaca774e3196210f3bdb442418)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:53.527763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93460cc-024f-4964-9b16-c3e50050f067 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:53.540725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 416/815] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:40:53 2022] PUT /identity/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/3ae6eeb3d4f94925a5e1dd497d352d2e/roles/ac03c5aaca774e3196210f3bdb442418 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:40:53.541697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93460cc-024f-4964-9b16-c3e50050f067 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:53.544908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b97b8-08c6-4f64-aae6-1ee3d7e66ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:53.545542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b97b8-08c6-4f64-aae6-1ee3d7e66ccc None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:53.545825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b97b8-08c6-4f64-aae6-1ee3d7e66ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:53.546140 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b97b8-08c6-4f64-aae6-1ee3d7e66ccc None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:53.547018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b97b8-08c6-4f64-aae6-1ee3d7e66ccc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:53.547325 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b97b8-08c6-4f64-aae6-1ee3d7e66ccc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:53.548789 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b97b8-08c6-4f64-aae6-1ee3d7e66ccc None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:53.570023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 400/816] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 3 12:40:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:53.571042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b97b8-08c6-4f64-aae6-1ee3d7e66ccc None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:53.628247 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:53.674030 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93460cc-024f-4964-9b16-c3e50050f067 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:53.684435 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b97b8-08c6-4f64-aae6-1ee3d7e66ccc None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:53.718342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c66809-58e0-4846-8d7f-119750f16e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:53.719037 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c66809-58e0-4846-8d7f-119750f16e04 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:53.719318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c66809-58e0-4846-8d7f-119750f16e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:53.719593 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c66809-58e0-4846-8d7f-119750f16e04 None admin] PATH_INFO: `/v3/services/network` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:53.720218 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c66809-58e0-4846-8d7f-119750f16e04 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:53.721578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c66809-58e0-4846-8d7f-119750f16e04 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:53.726243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5be0f-412c-46b5-834e-a137d902d5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:53.727099 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5be0f-412c-46b5-834e-a137d902d5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:53.727548 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5be0f-412c-46b5-834e-a137d902d5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:53.728059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd5be0f-412c-46b5-834e-a137d902d5e5 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:53.728935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5be0f-412c-46b5-834e-a137d902d5e5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:53.729401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5be0f-412c-46b5-834e-a137d902d5e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-d1c66809-58e0-4846-8d7f-119750f16e04 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:53.730619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:53.731825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:53.731825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:53.731825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:53.731825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:53.731825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:53.731825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:53.731825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:40:53.731825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:40:53.731825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:40:53.731825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:40:53.731825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:53.731825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:53.731825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:53.731825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:53.731825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:53.731825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:53.732766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:53.732766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:53.732766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:53.732766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:53.732766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:53.732766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:53.732766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:53.732766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:53.732766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:53.732766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:53.732766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:40:53.732766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:40:53.732766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 03 12:40:53.732766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:53.732766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 417/817] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:53 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:53.733619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c66809-58e0-4846-8d7f-119750f16e04 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:53.734068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd5be0f-412c-46b5-834e-a137d902d5e5 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:53.740960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 401/818] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 3 12:40:53 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:53.741796 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5be0f-412c-46b5-834e-a137d902d5e5 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:53.755056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:53.845362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c66809-58e0-4846-8d7f-119750f16e04 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:53.854939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd5be0f-412c-46b5-834e-a137d902d5e5 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:53.875975 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:53.885247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7187cf-0d4e-4493-8d96-4742266e9819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:53.885927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7187cf-0d4e-4493-8d96-4742266e9819 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:53.886217 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7187cf-0d4e-4493-8d96-4742266e9819 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:53.886517 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7187cf-0d4e-4493-8d96-4742266e9819 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:53.894003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7187cf-0d4e-4493-8d96-4742266e9819 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:53.895470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7187cf-0d4e-4493-8d96-4742266e9819 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:53.900020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76831aaf-9b7b-43df-8277-fdbc7868ebf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:53.900639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76831aaf-9b7b-43df-8277-fdbc7868ebf4 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:53.900920 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76831aaf-9b7b-43df-8277-fdbc7868ebf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:53.901190 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76831aaf-9b7b-43df-8277-fdbc7868ebf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:53.906952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-6d7187cf-0d4e-4493-8d96-4742266e9819 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:53.906952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:53.906952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:53.906952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:53.906952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:53.906952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:53.906952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:53.906952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:53.906952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:53.906952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:53.906952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:53.906952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:53.906952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:53.906952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:53.906952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:53.907862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:53.908797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:53.908797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:53.908797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:53.908797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:53.908797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:53.908797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:53.908797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:53.908797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:53.913090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 418/819] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:53.913915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7187cf-0d4e-4493-8d96-4742266e9819 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:53.974050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76831aaf-9b7b-43df-8277-fdbc7868ebf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:53.975579 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76831aaf-9b7b-43df-8277-fdbc7868ebf4 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:53.998744 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:54.038560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7187cf-0d4e-4493-8d96-4742266e9819 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:54.090202 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11266ba-689e-4c5b-96d0-3f60aecbb868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:54.091116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11266ba-689e-4c5b-96d0-3f60aecbb868 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:54.091549 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11266ba-689e-4c5b-96d0-3f60aecbb868 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:54.091971 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11266ba-689e-4c5b-96d0-3f60aecbb868 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:54.092938 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11266ba-689e-4c5b-96d0-3f60aecbb868 None admin] RBAC: Adding query filter params (name=network) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:54.093394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11266ba-689e-4c5b-96d0-3f60aecbb868 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:54.095377 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11266ba-689e-4c5b-96d0-3f60aecbb868 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:54.101321 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 419/820] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 3 12:40:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:54.102292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11266ba-689e-4c5b-96d0-3f60aecbb868 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:54.127080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 402/821] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:40:54.128082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76831aaf-9b7b-43df-8277-fdbc7868ebf4 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:54.143062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:54.239337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76831aaf-9b7b-43df-8277-fdbc7868ebf4 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:54.246435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11266ba-689e-4c5b-96d0-3f60aecbb868 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:54.288952 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7072ea-2c86-443a-a726-4f1019c95e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:54.290095 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7072ea-2c86-443a-a726-4f1019c95e62 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:54.290665 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7072ea-2c86-443a-a726-4f1019c95e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:54.291087 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a7d38-b142-473c-89e5-6717e586b788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:54.291331 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7072ea-2c86-443a-a726-4f1019c95e62 None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:54.291856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a7d38-b142-473c-89e5-6717e586b788 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:54.292208 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a7d38-b142-473c-89e5-6717e586b788 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:54.292552 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a7d38-b142-473c-89e5-6717e586b788 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:54.292867 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7072ea-2c86-443a-a726-4f1019c95e62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:54.293361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9a7d38-b142-473c-89e5-6717e586b788 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:54.293483 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7072ea-2c86-443a-a726-4f1019c95e62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:54.293825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9a7d38-b142-473c-89e5-6717e586b788 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:54.295304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9a7d38-b142-473c-89e5-6717e586b788 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:54.295564 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7072ea-2c86-443a-a726-4f1019c95e62 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:54.307262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 420/822] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 3 12:40:54 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:54.308248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a7d38-b142-473c-89e5-6717e586b788 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:54.316038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 403/823] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:54.316038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7072ea-2c86-443a-a726-4f1019c95e62 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:54.326207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:54.397199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:54.430685 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7072ea-2c86-443a-a726-4f1019c95e62 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:54.430885 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a7d38-b142-473c-89e5-6717e586b788 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:54.465467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1090c-e47f-4292-b635-afc25e698d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:54.466105 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1090c-e47f-4292-b635-afc25e698d45 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:54.466384 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1090c-e47f-4292-b635-afc25e698d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:54.466735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1090c-e47f-4292-b635-afc25e698d45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:54.469143 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64ccb8-1428-44d7-8d0d-e57ef73cc620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:54.469790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64ccb8-1428-44d7-8d0d-e57ef73cc620 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:54.470076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64ccb8-1428-44d7-8d0d-e57ef73cc620 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:54.470355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64ccb8-1428-44d7-8d0d-e57ef73cc620 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:54.471066 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64ccb8-1428-44d7-8d0d-e57ef73cc620 None admin] RBAC: Adding query filter params (type=network) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:54.471377 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64ccb8-1428-44d7-8d0d-e57ef73cc620 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:54.472738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64ccb8-1428-44d7-8d0d-e57ef73cc620 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:54.477462 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1090c-e47f-4292-b635-afc25e698d45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:54.482189 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 421/824] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 3 12:40:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:54.482189 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64ccb8-1428-44d7-8d0d-e57ef73cc620 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:54.482395 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a1090c-e47f-4292-b635-afc25e698d45 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:54.501640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-95a1090c-e47f-4292-b635-afc25e698d45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:54.501640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:54.501640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:54.501640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:54.501640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:54.501640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:54.501640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:54.501640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:54.501640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:54.501640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:54.501640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:54.501640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:54.501640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:54.501640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:54.501640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:54.502586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:54.503533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:54.503533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:54.503533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:54.503533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:54.503533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 404/825] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:54.503533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1090c-e47f-4292-b635-afc25e698d45 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:54.566460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:54.598969 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64ccb8-1428-44d7-8d0d-e57ef73cc620 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:54.618527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a1090c-e47f-4292-b635-afc25e698d45 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:54.635178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69383f14-408b-4f92-99a6-80c78553f12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:54.635798 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69383f14-408b-4f92-99a6-80c78553f12b None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:54.636079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69383f14-408b-4f92-99a6-80c78553f12b None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:54.636370 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69383f14-408b-4f92-99a6-80c78553f12b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:54.637272 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69383f14-408b-4f92-99a6-80c78553f12b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:54.638399 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69383f14-408b-4f92-99a6-80c78553f12b None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:54.644370 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:54.656425 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afc51e-3a65-49fc-a38f-021b59f75120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:54.657111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afc51e-3a65-49fc-a38f-021b59f75120 None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:54.657399 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afc51e-3a65-49fc-a38f-021b59f75120 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:54.657725 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afc51e-3a65-49fc-a38f-021b59f75120 None admin] PATH_INFO: `/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/cea54cb772344612aec4686d484de89f/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:54.659724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afc51e-3a65-49fc-a38f-021b59f75120 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0774035ad0f41c986771ef4a41c7d60, user_id=cea54cb772344612aec4686d484de89f, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:54.661186 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afc51e-3a65-49fc-a38f-021b59f75120 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:54.662635 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 422/826] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:40:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:40:54.664022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69383f14-408b-4f92-99a6-80c78553f12b None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:54.687736 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 405/827] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:40:54 2022] PUT /identity/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/cea54cb772344612aec4686d484de89f/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:40:54.688688 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afc51e-3a65-49fc-a38f-021b59f75120 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:54.802967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afc51e-3a65-49fc-a38f-021b59f75120 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:54.805434 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69383f14-408b-4f92-99a6-80c78553f12b None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:54.834334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:54.841811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f38d25-34d3-4aa2-bd88-3f724db89c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:54.843394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f38d25-34d3-4aa2-bd88-3f724db89c61 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:54.843730 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f38d25-34d3-4aa2-bd88-3f724db89c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:54.844008 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f38d25-34d3-4aa2-bd88-3f724db89c61 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:54.844731 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f38d25-34d3-4aa2-bd88-3f724db89c61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:54.845036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f38d25-34d3-4aa2-bd88-3f724db89c61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:54.850906 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f38d25-34d3-4aa2-bd88-3f724db89c61 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:54.861407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcfcd9-6a7e-4ba2-ba98-83a8fa9168f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:54.862867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcfcd9-6a7e-4ba2-ba98-83a8fa9168f4 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:54.863359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcfcd9-6a7e-4ba2-ba98-83a8fa9168f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:54.863881 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcfcd9-6a7e-4ba2-ba98-83a8fa9168f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:54.865635 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 406/828] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:54.865951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fcfcd9-6a7e-4ba2-ba98-83a8fa9168f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:54.867699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fcfcd9-6a7e-4ba2-ba98-83a8fa9168f4 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:54.871608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f38d25-34d3-4aa2-bd88-3f724db89c61 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:54.901124 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 423/829] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:40:54 2022] POST /identity/v3/endpoints => generated 337 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:40:54.945907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:55.014234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f38d25-34d3-4aa2-bd88-3f724db89c61 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:55.061880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8d00e-ead1-45fb-8f6f-e892b62e1f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:55.062645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8d00e-ead1-45fb-8f6f-e892b62e1f42 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:55.063031 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8d00e-ead1-45fb-8f6f-e892b62e1f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:55.063409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8d00e-ead1-45fb-8f6f-e892b62e1f42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:55.071566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8d00e-ead1-45fb-8f6f-e892b62e1f42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:55.073189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c8d00e-ead1-45fb-8f6f-e892b62e1f42 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:55.086468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-f7c8d00e-ead1-45fb-8f6f-e892b62e1f42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:55.086468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:55.086468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:55.086468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:55.086468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:55.086468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:55.086468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:55.086468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:55.086468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:55.086468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:55.086468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:55.086468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:55.086468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:55.086468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:55.086468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:55.087669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:55.088773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:55.088773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:55.088773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:55.088773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:55.088773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:55.088773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:55.088773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:55.088773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:55.090671 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 407/830] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:55.096796 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcfcd9-6a7e-4ba2-ba98-83a8fa9168f4 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:55.203736 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcfcd9-6a7e-4ba2-ba98-83a8fa9168f4 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:55.239825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf90d60-cd63-4d27-83f1-f5ef24da0639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:55.240622 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf90d60-cd63-4d27-83f1-f5ef24da0639 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:55.241002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf90d60-cd63-4d27-83f1-f5ef24da0639 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:55.241406 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf90d60-cd63-4d27-83f1-f5ef24da0639 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:55.242287 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf90d60-cd63-4d27-83f1-f5ef24da0639 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:55.242728 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf90d60-cd63-4d27-83f1-f5ef24da0639 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:55.244163 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf90d60-cd63-4d27-83f1-f5ef24da0639 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:55.257522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 424/831] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:55.263094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8d00e-ead1-45fb-8f6f-e892b62e1f42 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:55.375975 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:55.383370 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8d00e-ead1-45fb-8f6f-e892b62e1f42 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:55.420290 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19cdf9a-2cf3-4aba-8a4a-e701e7f716eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:55.420952 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19cdf9a-2cf3-4aba-8a4a-e701e7f716eb None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:55.421242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19cdf9a-2cf3-4aba-8a4a-e701e7f716eb None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:55.421516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19cdf9a-2cf3-4aba-8a4a-e701e7f716eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:55.422316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19cdf9a-2cf3-4aba-8a4a-e701e7f716eb None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=7b634f567dca42dc95df175a4f36c1d1) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:55.422655 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19cdf9a-2cf3-4aba-8a4a-e701e7f716eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:55.424098 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19cdf9a-2cf3-4aba-8a4a-e701e7f716eb None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:55.430860 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 408/832] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:40:55 2022] GET /identity/v3/role_assignments?user.id=7b634f567dca42dc95df175a4f36c1d1&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:55.604546 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31cec1-956a-470f-b2ca-f8db8de6d95b None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:55.604878 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31cec1-956a-470f-b2ca-f8db8de6d95b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:55.605108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31cec1-956a-470f-b2ca-f8db8de6d95b None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:55.606603 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 425/833] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:55.612233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d184ac-0938-4005-885e-2ca71fdb06da None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:55.612519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d184ac-0938-4005-885e-2ca71fdb06da None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:55.612793 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d184ac-0938-4005-885e-2ca71fdb06da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:55.634151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-72d184ac-0938-4005-885e-2ca71fdb06da None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:55.635972 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-72d184ac-0938-4005-885e-2ca71fdb06da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:55.708612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 409/834] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:40:55.730954 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31cec1-956a-470f-b2ca-f8db8de6d95b None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:55.773748 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:55.838697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31cec1-956a-470f-b2ca-f8db8de6d95b None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:55.874814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254bb13-dc67-404c-823b-c1ca05269b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:55.875546 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254bb13-dc67-404c-823b-c1ca05269b93 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:55.875744 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254bb13-dc67-404c-823b-c1ca05269b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:55.876028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2254bb13-dc67-404c-823b-c1ca05269b93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:55.886692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2254bb13-dc67-404c-823b-c1ca05269b93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:55.891864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-2254bb13-dc67-404c-823b-c1ca05269b93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:55.891864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:55.891864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:55.891864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:55.891864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:55.891864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:55.891864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:55.891864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:55.891864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:55.891864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:55.891864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:55.891864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:55.891864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:55.891864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:55.891864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:55.891864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:55.892807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:55.892807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:55.892807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:55.892807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:55.892807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:55.892807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:55.892807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:55.892807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:55.892807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:55.892807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:55.892807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:55.892807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:55.892807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:55.892807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:55.892807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:55.892807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:55.893651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:55.893651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:55.893651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:55.893651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:55.893651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:55.893651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:55.893651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:55.893651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:55.893651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:55.893651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:55.893651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:40:55.893651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:55.893651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 426/835] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:40:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:55.898924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d184ac-0938-4005-885e-2ca71fdb06da None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:55.994990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:56.003837 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d184ac-0938-4005-885e-2ca71fdb06da None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:56.039109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3dfe8-5f08-4091-8da7-2d264770d063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:56.040085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3dfe8-5f08-4091-8da7-2d264770d063 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:56.040588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3dfe8-5f08-4091-8da7-2d264770d063 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:56.041073 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3dfe8-5f08-4091-8da7-2d264770d063 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:56.041996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3dfe8-5f08-4091-8da7-2d264770d063 None admin] RBAC: Adding query filter params (name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:56.042591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3dfe8-5f08-4091-8da7-2d264770d063 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:56.044163 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3dfe8-5f08-4091-8da7-2d264770d063 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:56.053701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 410/836] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:40:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:56.058932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254bb13-dc67-404c-823b-c1ca05269b93 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:56.162035 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:56.166448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2254bb13-dc67-404c-823b-c1ca05269b93 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:56.205761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10db0a-fdd0-4196-9ff0-c2b78c8c0ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:56.206416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10db0a-fdd0-4196-9ff0-c2b78c8c0ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:56.206721 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10db0a-fdd0-4196-9ff0-c2b78c8c0ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:56.206997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10db0a-fdd0-4196-9ff0-c2b78c8c0ec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:56.217333 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10db0a-fdd0-4196-9ff0-c2b78c8c0ec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:56.219310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10db0a-fdd0-4196-9ff0-c2b78c8c0ec0 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:56.234750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-5f10db0a-fdd0-4196-9ff0-c2b78c8c0ec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:56.234750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:56.234750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:56.234750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:56.234750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:56.234750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:56.234750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:56.234750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:56.234750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:56.234750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:56.234750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:56.234750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:56.234750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:56.234750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:56.234750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:56.235657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:56.236561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:56.236561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:56.236561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:56.236561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:56.236561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 427/837] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:56.241693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3dfe8-5f08-4091-8da7-2d264770d063 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:56.348576 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3dfe8-5f08-4091-8da7-2d264770d063 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:56.383056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a09c71-b82f-4775-9972-7b629ff3e206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:56.383897 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a09c71-b82f-4775-9972-7b629ff3e206 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:56.384352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a09c71-b82f-4775-9972-7b629ff3e206 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:56.384796 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a09c71-b82f-4775-9972-7b629ff3e206 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:56.385684 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a09c71-b82f-4775-9972-7b629ff3e206 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:56.386161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a09c71-b82f-4775-9972-7b629ff3e206 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:56.387733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a09c71-b82f-4775-9972-7b629ff3e206 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:56.400069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 411/838] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:56.406510 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10db0a-fdd0-4196-9ff0-c2b78c8c0ec0 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:56.511448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10db0a-fdd0-4196-9ff0-c2b78c8c0ec0 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:56.548416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4852224-f70c-44b1-b672-e5b1df9cc9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:56.549067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4852224-f70c-44b1-b672-e5b1df9cc9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:56.549354 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4852224-f70c-44b1-b672-e5b1df9cc9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:56.549702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4852224-f70c-44b1-b672-e5b1df9cc9a3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:56.557019 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4852224-f70c-44b1-b672-e5b1df9cc9a3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:56.558446 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4852224-f70c-44b1-b672-e5b1df9cc9a3 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:56.570692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-e4852224-f70c-44b1-b672-e5b1df9cc9a3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 12:40:56.570692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:56.570692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:56.570692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:56.570692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:56.570692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:56.570692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:56.570692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:56.570692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:56.570692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:56.570692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:56.570692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:56.570692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:56.570692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:56.570692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:56.570692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:56.571766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:56.572909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:56.572909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:56.572909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:56.572909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:56.572909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:56.572909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:56.572909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:56.572909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:56.572909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 12:40:56.572909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:56.572909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 428/839] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 3 12:40:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:56.578198 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a09c71-b82f-4775-9972-7b629ff3e206 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:56.597269 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:56.686655 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a09c71-b82f-4775-9972-7b629ff3e206 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:56.723734 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd49f8e-2a7f-4218-9fb5-93f296c1c3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:56.724656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd49f8e-2a7f-4218-9fb5-93f296c1c3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:56.725116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd49f8e-2a7f-4218-9fb5-93f296c1c3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:56.725644 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd49f8e-2a7f-4218-9fb5-93f296c1c3a9 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:56.726684 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd49f8e-2a7f-4218-9fb5-93f296c1c3a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:56.727173 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd49f8e-2a7f-4218-9fb5-93f296c1c3a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:56.728766 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd49f8e-2a7f-4218-9fb5-93f296c1c3a9 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:56.750600 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 412/840] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 3 12:40:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:56.756276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4852224-f70c-44b1-b672-e5b1df9cc9a3 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:56.770914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:56.864382 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4852224-f70c-44b1-b672-e5b1df9cc9a3 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:56.898563 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86fbf4-cdd6-467e-b4d9-aa49c67847d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:56.899177 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86fbf4-cdd6-467e-b4d9-aa49c67847d5 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:56.899501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86fbf4-cdd6-467e-b4d9-aa49c67847d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:56.899787 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86fbf4-cdd6-467e-b4d9-aa49c67847d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:56.969051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86fbf4-cdd6-467e-b4d9-aa49c67847d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:56.970508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86fbf4-cdd6-467e-b4d9-aa49c67847d5 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:56.982026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:57.038836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fc2be-04a4-4968-892f-dfe2e0fcfd12 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:57.039343 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fc2be-04a4-4968-892f-dfe2e0fcfd12 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:57.039343 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fc2be-04a4-4968-892f-dfe2e0fcfd12 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:57.040751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 413/841] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:57.047243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777766b-4994-4449-9f11-e6d6adc2c3d3 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:57.047771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777766b-4994-4449-9f11-e6d6adc2c3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:57.048224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777766b-4994-4449-9f11-e6d6adc2c3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:57.072773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-0777766b-4994-4449-9f11-e6d6adc2c3d3 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:57.075321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-0777766b-4994-4449-9f11-e6d6adc2c3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:57.078354 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 429/842] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:56 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:40:57.079691 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ba2a3-41fe-4cfc-ad96-07b26f505c97 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:57.079980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ba2a3-41fe-4cfc-ad96-07b26f505c97 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:57.080261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ba2a3-41fe-4cfc-ad96-07b26f505c97 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:57.081558 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 430/843] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:57.087195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ba2a3-41fe-4cfc-ad96-07b26f505c97 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:57.160479 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 414/844] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:40:57.166724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25754b-9e92-4ffa-8008-c952157e6516 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:57.167458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25754b-9e92-4ffa-8008-c952157e6516 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:57.168405 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25754b-9e92-4ffa-8008-c952157e6516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:57.178443 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:57.193872 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-9c25754b-9e92-4ffa-8008-c952157e6516 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:57.194363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ba2a3-41fe-4cfc-ad96-07b26f505c97 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:57.195078 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-9c25754b-9e92-4ffa-8008-c952157e6516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:57.206818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:57.232127 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3b0f8-f9a0-45d8-8afe-ce9732a4ef05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:57.232814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3b0f8-f9a0-45d8-8afe-ce9732a4ef05 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:57.233096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3b0f8-f9a0-45d8-8afe-ce9732a4ef05 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:57.233389 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3b0f8-f9a0-45d8-8afe-ce9732a4ef05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:57.245506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3b0f8-f9a0-45d8-8afe-ce9732a4ef05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:57.247083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3b0f8-f9a0-45d8-8afe-ce9732a4ef05 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:57.263361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-60f3b0f8-f9a0-45d8-8afe-ce9732a4ef05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:57.263361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:57.263361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:57.263361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:57.263361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:57.263361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:57.263361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:57.263361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:57.263361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:57.263361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:57.263361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:57.263361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:57.263361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:57.263361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:57.263361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:57.264293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:57.265800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:57.265800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:57.265800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:57.265800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:57.265800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 431/845] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:57.266103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3b0f8-f9a0-45d8-8afe-ce9732a4ef05 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:57.271949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 415/846] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:40:57.272657 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25754b-9e92-4ffa-8008-c952157e6516 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:57.377864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25754b-9e92-4ffa-8008-c952157e6516 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:57.383223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3b0f8-f9a0-45d8-8afe-ce9732a4ef05 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:57.417090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a955f67-d4a7-4fed-b920-57fc095c6d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:57.417732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a955f67-d4a7-4fed-b920-57fc095c6d20 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:57.418013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a955f67-d4a7-4fed-b920-57fc095c6d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:57.418293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a955f67-d4a7-4fed-b920-57fc095c6d20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:57.422071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ec4da-ffe7-41fa-9554-db527015341f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:57.422949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ec4da-ffe7-41fa-9554-db527015341f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:57.423592 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ec4da-ffe7-41fa-9554-db527015341f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:57.424203 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ec4da-ffe7-41fa-9554-db527015341f None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:57.425133 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ec4da-ffe7-41fa-9554-db527015341f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:57.425871 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ec4da-ffe7-41fa-9554-db527015341f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:57.427882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ec4da-ffe7-41fa-9554-db527015341f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:57.429261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a955f67-d4a7-4fed-b920-57fc095c6d20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:40:57.430214 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:57.434921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-3a955f67-d4a7-4fed-b920-57fc095c6d20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:57.434921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:57.434921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:57.434921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:57.434921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:57.434921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:57.434921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:57.434921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:57.434921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:57.434921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:57.434921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:57.434921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:57.434921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:40:57.434921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:40:57.434921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:40:57.434921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:40:57.435884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:40:57.435884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:40:57.435884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:40:57.435884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:40:57.435884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:57.435884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:57.435884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:57.435884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:57.435884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:57.435884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:57.435884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:57.435884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:57.435884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:57.435884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:57.435884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:57.435884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:57.436834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:57.436834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:57.436834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:57.436834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:57.436834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:40:57.436834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:40:57.436834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:40:57.436834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:40:57.436834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:40:57.436834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:40:57.436834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:40:57.436834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:57.436834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 432/847] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:40:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:57.439167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a955f67-d4a7-4fed-b920-57fc095c6d20 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:57.451814 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 416/848] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:57.451814 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ec4da-ffe7-41fa-9554-db527015341f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:57.465980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:57.547901 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a955f67-d4a7-4fed-b920-57fc095c6d20 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:57.559708 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ec4da-ffe7-41fa-9554-db527015341f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:57.582452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc157e37-53e1-4ea2-baa3-29434aead853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:57.583125 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc157e37-53e1-4ea2-baa3-29434aead853 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:57.583409 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc157e37-53e1-4ea2-baa3-29434aead853 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:57.583691 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc157e37-53e1-4ea2-baa3-29434aead853 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:57.584320 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc157e37-53e1-4ea2-baa3-29434aead853 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:57.585709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc157e37-53e1-4ea2-baa3-29434aead853 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-dc157e37-53e1-4ea2-baa3-29434aead853 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:57.594617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:57.595818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:57.595818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:57.595818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:57.595818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:57.595818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:57.595818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:57.595818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:40:57.595818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:40:57.595818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:40:57.595818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:40:57.595818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:57.595818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:57.595818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:57.595818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:57.595818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:57.595818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:57.596670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:57.596670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:57.596670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:57.596670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:57.596670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:57.596670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:57.596670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:57.596670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:57.596670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:57.596670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:57.596670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:40:57.596670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:40:57.596670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 12:40:57.596670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:57.596670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 433/849] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:40:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:57.597466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc157e37-53e1-4ea2-baa3-29434aead853 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:57.602007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb3018-d1ee-4630-b04f-f4a256a191ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:57.602828 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb3018-d1ee-4630-b04f-f4a256a191ff None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:57.603282 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb3018-d1ee-4630-b04f-f4a256a191ff None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:57.603800 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb3018-d1ee-4630-b04f-f4a256a191ff None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:57.604804 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb3018-d1ee-4630-b04f-f4a256a191ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:57.605114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb3018-d1ee-4630-b04f-f4a256a191ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:57.606494 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb3018-d1ee-4630-b04f-f4a256a191ff None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:57.615630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 417/850] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:40:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:57.616420 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb3018-d1ee-4630-b04f-f4a256a191ff None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:57.669887 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:57.685963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:57.701429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc157e37-53e1-4ea2-baa3-29434aead853 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:57.713528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb3018-d1ee-4630-b04f-f4a256a191ff None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:57.734530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42538f43-75f1-458c-8251-3a71971a4919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:57.735167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42538f43-75f1-458c-8251-3a71971a4919 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:57.735446 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42538f43-75f1-458c-8251-3a71971a4919 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:57.735721 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42538f43-75f1-458c-8251-3a71971a4919 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:57.742251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42538f43-75f1-458c-8251-3a71971a4919 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:57.743776 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42538f43-75f1-458c-8251-3a71971a4919 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:57.747047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d0c00-9010-46db-8097-eee94af5a576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:57.747652 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d0c00-9010-46db-8097-eee94af5a576 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:57.747926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d0c00-9010-46db-8097-eee94af5a576 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:57.748195 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d0c00-9010-46db-8097-eee94af5a576 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:57.748937 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d0c00-9010-46db-8097-eee94af5a576 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:57.749273 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d0c00-9010-46db-8097-eee94af5a576 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:57.750642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8d0c00-9010-46db-8097-eee94af5a576 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:57.755135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-42538f43-75f1-458c-8251-3a71971a4919 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:57.755135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:57.755135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:57.755135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:57.755135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:57.755135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:57.755135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:57.755135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:57.755135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:57.755135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:57.755135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:57.755135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:57.755135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:57.755135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:57.755135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:57.756008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:57.756919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:57.756919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:57.756919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:57.756919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:57.756919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:57.756919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:57.756919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:57.756919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:57.757357 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 418/851] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:40:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:57.757429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 434/852] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:57.757496 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a5fe0-41c9-4925-93c1-1f282e67a176 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:57.757496 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a5fe0-41c9-4925-93c1-1f282e67a176 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:57.757496 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934a5fe0-41c9-4925-93c1-1f282e67a176 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:57.757683 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42538f43-75f1-458c-8251-3a71971a4919 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:57.761779 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 419/853] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:57.762559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a5fe0-41c9-4925-93c1-1f282e67a176 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:57.860288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42538f43-75f1-458c-8251-3a71971a4919 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:57.866126 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934a5fe0-41c9-4925-93c1-1f282e67a176 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:57.888142 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:57.895493 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170ed64-0afa-47d8-83ab-6076f6734bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:57.896137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170ed64-0afa-47d8-83ab-6076f6734bdf None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:57.896414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170ed64-0afa-47d8-83ab-6076f6734bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:57.896687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170ed64-0afa-47d8-83ab-6076f6734bdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:57.902357 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c14e1-ccbb-40cd-91e1-b94735879de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:57.903003 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c14e1-ccbb-40cd-91e1-b94735879de8 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:57.903277 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c14e1-ccbb-40cd-91e1-b94735879de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:57.903555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c14e1-ccbb-40cd-91e1-b94735879de8 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:57.904234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c14e1-ccbb-40cd-91e1-b94735879de8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:57.904538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c14e1-ccbb-40cd-91e1-b94735879de8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:57.906435 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c14e1-ccbb-40cd-91e1-b94735879de8 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:57.907522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170ed64-0afa-47d8-83ab-6076f6734bdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:57.908935 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170ed64-0afa-47d8-83ab-6076f6734bdf None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:57.911039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 420/854] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:40:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:57.912204 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7900b-78f8-4c80-b041-62526431a3fb None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:57.912474 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7900b-78f8-4c80-b041-62526431a3fb None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:57.912804 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7900b-78f8-4c80-b041-62526431a3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:57.924309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-1170ed64-0afa-47d8-83ab-6076f6734bdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:57.924309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:57.924309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:57.924309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:57.924309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:57.924309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:57.924309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:57.924309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:57.924309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:57.924309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:57.924309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:57.924309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:57.924309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:57.924309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:57.924309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:57.925270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:57.926179 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:57.926179 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:57.926179 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:57.926179 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:57.927747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:57.930364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 435/855] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:57.931377 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170ed64-0afa-47d8-83ab-6076f6734bdf None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:57.933331 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-61b7900b-78f8-4c80-b041-62526431a3fb None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:57.935009 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-61b7900b-78f8-4c80-b041-62526431a3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:40:58.014874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 421/856] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:40:58.015872 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7900b-78f8-4c80-b041-62526431a3fb None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:58.048585 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170ed64-0afa-47d8-83ab-6076f6734bdf None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:58.085185 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a8120-2cee-4357-a2fc-b04a9ca4ee9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:58.086361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a8120-2cee-4357-a2fc-b04a9ca4ee9d None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:58.086808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a8120-2cee-4357-a2fc-b04a9ca4ee9d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:58.087165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a8120-2cee-4357-a2fc-b04a9ca4ee9d None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:58.088113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a8120-2cee-4357-a2fc-b04a9ca4ee9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:58.088415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a8120-2cee-4357-a2fc-b04a9ca4ee9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:58.089928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a8120-2cee-4357-a2fc-b04a9ca4ee9d None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:58.109495 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 436/857] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:58.109495 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a8120-2cee-4357-a2fc-b04a9ca4ee9d None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:58.134102 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7900b-78f8-4c80-b041-62526431a3fb None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:58.150299 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:58.167913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:58.174597 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fafee-be6c-4afb-9a50-953747eaa1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:58.175258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fafee-be6c-4afb-9a50-953747eaa1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:58.175604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fafee-be6c-4afb-9a50-953747eaa1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:58.175949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fafee-be6c-4afb-9a50-953747eaa1a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:58.176830 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317fafee-be6c-4afb-9a50-953747eaa1a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:58.177195 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317fafee-be6c-4afb-9a50-953747eaa1a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:58.178770 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317fafee-be6c-4afb-9a50-953747eaa1a6 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:58.192859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 422/858] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:58.193854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fafee-be6c-4afb-9a50-953747eaa1a6 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:58.218230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a8120-2cee-4357-a2fc-b04a9ca4ee9d None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:58.254148 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e7e53-6aa7-4cf8-b175-a7b72717baea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:58.254817 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e7e53-6aa7-4cf8-b175-a7b72717baea None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:58.255105 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e7e53-6aa7-4cf8-b175-a7b72717baea None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:58.255373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e7e53-6aa7-4cf8-b175-a7b72717baea None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:58.255996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e7e53-6aa7-4cf8-b175-a7b72717baea None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:58.257344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e7e53-6aa7-4cf8-b175-a7b72717baea None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:58.266791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-e92e7e53-6aa7-4cf8-b175-a7b72717baea None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 12:40:58.266791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:58.266791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:40:58.266791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:40:58.266791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:40:58.266791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:40:58.266791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:40:58.266791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:40:58.266791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 12:40:58.266791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:58.266791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:40:58.266791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:58.266791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:58.266791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:58.266791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:58.266791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:58.267873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:58.267873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:58.267873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:58.267873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:58.267873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:58.267873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:58.267873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:58.267873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:40:58.267873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:40:58.267873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:40:58.267873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:40:58.267873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:58.267873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:58.267873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:58.267873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:58.267873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:58.268753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:58.268753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:58.268753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:58.268753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:58.268753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:58.268753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:58.268753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:58.268753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:58.268753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:58.268753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:58.268753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:58.268753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:40:58.268753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:40:58.268753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 12:40:58.268753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:58.268753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 437/859] 10.222.0.65 () {56 vars in 1132 bytes} [Wed Aug 3 12:40:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:58.269617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e7e53-6aa7-4cf8-b175-a7b72717baea None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:58.302082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fafee-be6c-4afb-9a50-953747eaa1a6 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:58.340250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00aa2c-e211-4b43-be4f-7cbbb6ba470b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:58.341020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00aa2c-e211-4b43-be4f-7cbbb6ba470b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:58.341389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00aa2c-e211-4b43-be4f-7cbbb6ba470b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:58.341764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00aa2c-e211-4b43-be4f-7cbbb6ba470b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:58.342601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00aa2c-e211-4b43-be4f-7cbbb6ba470b None admin] RBAC: Adding query filter params (role.id=ac03c5aaca774e3196210f3bdb442418, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=3ae6eeb3d4f94925a5e1dd497d352d2e) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:58.342940 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00aa2c-e211-4b43-be4f-7cbbb6ba470b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:58.344395 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00aa2c-e211-4b43-be4f-7cbbb6ba470b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:58.351229 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 423/860] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:40:58 2022] GET /identity/v3/role_assignments?user.id=3ae6eeb3d4f94925a5e1dd497d352d2e&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=ac03c5aaca774e3196210f3bdb442418 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:58.352021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00aa2c-e211-4b43-be4f-7cbbb6ba470b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:58.388979 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e7e53-6aa7-4cf8-b175-a7b72717baea None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:58.410391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:58.417803 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:58.434667 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44438308-d6d2-4f9f-8e11-bb5b9723be10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:58.435751 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44438308-d6d2-4f9f-8e11-bb5b9723be10 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:58.437982 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44438308-d6d2-4f9f-8e11-bb5b9723be10 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:58.438424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44438308-d6d2-4f9f-8e11-bb5b9723be10 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:58.448032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44438308-d6d2-4f9f-8e11-bb5b9723be10 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:58.449778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44438308-d6d2-4f9f-8e11-bb5b9723be10 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:58.463332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-44438308-d6d2-4f9f-8e11-bb5b9723be10 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 03 12:40:58.463332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:58.463332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:58.463332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:58.463332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:58.463332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:58.463332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:58.463332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:58.463332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:58.463332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:58.463332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:58.463332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:58.463332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:40:58.463332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:40:58.463332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:40:58.463332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:58.464391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:58.465350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:58.465350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:58.465350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:40:58.465350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:40:58.465350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:40:58.465350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:40:58.465350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:40:58.465350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:40:58.465350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 03 12:40:58.465350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:40:58.468817 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 438/861] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:40:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:58.474268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44438308-d6d2-4f9f-8e11-bb5b9723be10 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:58.503112 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00aa2c-e211-4b43-be4f-7cbbb6ba470b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:58.547500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc902c-c942-4dd4-b1e5-55a65c17e87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:58.548229 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc902c-c942-4dd4-b1e5-55a65c17e87a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:58.548591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc902c-c942-4dd4-b1e5-55a65c17e87a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:58.548935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc902c-c942-4dd4-b1e5-55a65c17e87a None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:58.549691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defc902c-c942-4dd4-b1e5-55a65c17e87a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:58.550063 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defc902c-c942-4dd4-b1e5-55a65c17e87a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:58.551564 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defc902c-c942-4dd4-b1e5-55a65c17e87a None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:58.558659 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 424/862] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 3 12:40:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:58.564416 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc902c-c942-4dd4-b1e5-55a65c17e87a None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:58.595299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44438308-d6d2-4f9f-8e11-bb5b9723be10 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:58.642220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81475916-2e31-4e55-9869-a3725d390e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:58.642888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81475916-2e31-4e55-9869-a3725d390e88 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:58.643165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81475916-2e31-4e55-9869-a3725d390e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:58.643438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81475916-2e31-4e55-9869-a3725d390e88 None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:58.644232 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81475916-2e31-4e55-9869-a3725d390e88 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:58.644540 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81475916-2e31-4e55-9869-a3725d390e88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:58.645948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81475916-2e31-4e55-9869-a3725d390e88 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:58.665195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 439/863] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 3 12:40:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:40:58.670749 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81475916-2e31-4e55-9869-a3725d390e88 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:58.690698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc902c-c942-4dd4-b1e5-55a65c17e87a None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:58.693015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:58.714934 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:58.736561 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f5967-f3d1-405f-a450-b4497cb1b02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:58.737752 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f5967-f3d1-405f-a450-b4497cb1b02a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:58.738676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f5967-f3d1-405f-a450-b4497cb1b02a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:58.739437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f5967-f3d1-405f-a450-b4497cb1b02a None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:58.740569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f5967-f3d1-405f-a450-b4497cb1b02a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:58.741453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f5967-f3d1-405f-a450-b4497cb1b02a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:58.743338 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07f5967-f3d1-405f-a450-b4497cb1b02a None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:58.750639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 425/864] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 3 12:40:58 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:58.792587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81475916-2e31-4e55-9869-a3725d390e88 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:58.832575 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a57b6-9201-4b82-8898-83e4ba5ca8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:58.833212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a57b6-9201-4b82-8898-83e4ba5ca8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:58.833490 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a57b6-9201-4b82-8898-83e4ba5ca8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:58.833761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a57b6-9201-4b82-8898-83e4ba5ca8c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:58.911151 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a57b6-9201-4b82-8898-83e4ba5ca8c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:58.911776 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a57b6-9201-4b82-8898-83e4ba5ca8c7 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:58.955092 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:59.016809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 440/865] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:40:58 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:40:59.022331 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f5967-f3d1-405f-a450-b4497cb1b02a None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:59.131787 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f5967-f3d1-405f-a450-b4497cb1b02a None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:59.171671 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59818fbf-c3e1-47b8-997e-36fbb41a0f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:59.172311 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59818fbf-c3e1-47b8-997e-36fbb41a0f3f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:59.172580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59818fbf-c3e1-47b8-997e-36fbb41a0f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:59.172853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59818fbf-c3e1-47b8-997e-36fbb41a0f3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:59.183620 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59818fbf-c3e1-47b8-997e-36fbb41a0f3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:59.185317 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59818fbf-c3e1-47b8-997e-36fbb41a0f3f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:59.199888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-59818fbf-c3e1-47b8-997e-36fbb41a0f3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:59.199888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:59.199888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:59.199888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:59.199888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:59.199888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:59.199888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:59.199888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:59.199888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:59.199888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:59.199888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:59.199888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:59.199888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:40:59.199888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:40:59.199888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:59.201042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:59.202701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:40:59.202701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:40:59.202701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:40:59.202701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:59.202701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 426/866] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:40:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:59.208872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a57b6-9201-4b82-8898-83e4ba5ca8c7 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:59.318218 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a57b6-9201-4b82-8898-83e4ba5ca8c7 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:59.353283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fd563-21a2-4a43-b190-55e63e6edf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:59.354292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fd563-21a2-4a43-b190-55e63e6edf03 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:59.354877 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fd563-21a2-4a43-b190-55e63e6edf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:59.355410 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fd563-21a2-4a43-b190-55e63e6edf03 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:59.356290 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fd563-21a2-4a43-b190-55e63e6edf03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:59.356802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fd563-21a2-4a43-b190-55e63e6edf03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:59.358513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144fd563-21a2-4a43-b190-55e63e6edf03 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:59.367328 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:59.370965 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 441/867] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:59.377085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59818fbf-c3e1-47b8-997e-36fbb41a0f3f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:59.397506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:59.483122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59818fbf-c3e1-47b8-997e-36fbb41a0f3f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:59.519089 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f464d2-6b8f-43fa-a6d7-628b511bdc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:59.520130 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f464d2-6b8f-43fa-a6d7-628b511bdc66 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:59.520615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f464d2-6b8f-43fa-a6d7-628b511bdc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:59.521091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f464d2-6b8f-43fa-a6d7-628b511bdc66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:59.528193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f464d2-6b8f-43fa-a6d7-628b511bdc66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:59.529902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f464d2-6b8f-43fa-a6d7-628b511bdc66 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:59.541716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-76f464d2-6b8f-43fa-a6d7-628b511bdc66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:59.541716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:40:59.541716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:40:59.541716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:40:59.541716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:40:59.541716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:40:59.541716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:40:59.541716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:40:59.541716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:40:59.541716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:40:59.541716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:40:59.541716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:40:59.541716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:40:59.541716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:40:59.541716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:40:59.542650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:40:59.543569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:40:59.543569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:40:59.543569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:40:59.543569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:40:59.543569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:40:59.543569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:40:59.543569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:40:59.543569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:40:59.545947 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 427/868] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:40:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:40:59.554166 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fd563-21a2-4a43-b190-55e63e6edf03 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:59.658360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fd563-21a2-4a43-b190-55e63e6edf03 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:59.693339 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c0b8c-29ec-4a8e-998a-b4f92c7e17f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:59.693961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c0b8c-29ec-4a8e-998a-b4f92c7e17f0 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:59.694235 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c0b8c-29ec-4a8e-998a-b4f92c7e17f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:59.694523 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c0b8c-29ec-4a8e-998a-b4f92c7e17f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:59.695270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198c0b8c-29ec-4a8e-998a-b4f92c7e17f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:59.695602 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198c0b8c-29ec-4a8e-998a-b4f92c7e17f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:59.696994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198c0b8c-29ec-4a8e-998a-b4f92c7e17f0 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:59.709748 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 442/869] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:40:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:59.715641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f464d2-6b8f-43fa-a6d7-628b511bdc66 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:40:59.796597 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:40:59.819607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f464d2-6b8f-43fa-a6d7-628b511bdc66 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:40:59.855652 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29e41d-33f8-4f16-a81b-217c63d8e959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:40:59.856276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29e41d-33f8-4f16-a81b-217c63d8e959 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:59.856562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29e41d-33f8-4f16-a81b-217c63d8e959 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:59.859033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29e41d-33f8-4f16-a81b-217c63d8e959 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:59.859033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29e41d-33f8-4f16-a81b-217c63d8e959 None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=cea54cb772344612aec4686d484de89f) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:40:59.859033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29e41d-33f8-4f16-a81b-217c63d8e959 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:40:59.860677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29e41d-33f8-4f16-a81b-217c63d8e959 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:40:59.866650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 428/870] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:40:59 2022] GET /identity/v3/role_assignments?user.id=cea54cb772344612aec4686d484de89f&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:40:59.958458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85a0bf-2529-4801-9cb4-e64a301deaa3 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:59.959017 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85a0bf-2529-4801-9cb4-e64a301deaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:59.959145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85a0bf-2529-4801-9cb4-e64a301deaa3 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:59.960724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 443/871] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:40:59.966406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4f572-445b-4d25-8d24-199ce150ae90 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:40:59.966723 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4f572-445b-4d25-8d24-199ce150ae90 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:40:59.967004 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b4f572-445b-4d25-8d24-199ce150ae90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:40:59.987336 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-47b4f572-445b-4d25-8d24-199ce150ae90 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:40:59.988568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-47b4f572-445b-4d25-8d24-199ce150ae90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:00.063159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 429/872] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:40:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:00.087142 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85a0bf-2529-4801-9cb4-e64a301deaa3 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:00.150228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:00.195742 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85a0bf-2529-4801-9cb4-e64a301deaa3 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:00.230807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e53d2-4734-427f-b35a-492ef005d8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:00.231431 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e53d2-4734-427f-b35a-492ef005d8e2 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:00.231713 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e53d2-4734-427f-b35a-492ef005d8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:00.231990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e53d2-4734-427f-b35a-492ef005d8e2 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:00.232682 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e53d2-4734-427f-b35a-492ef005d8e2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:00.234658 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e53d2-4734-427f-b35a-492ef005d8e2 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:00.251588 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 444/873] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 3 12:41:00 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:41:00.581120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb11ec-cb50-4d30-8b4e-36ab4c5d2840 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:00.581662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb11ec-cb50-4d30-8b4e-36ab4c5d2840 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:00.581662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bb11ec-cb50-4d30-8b4e-36ab4c5d2840 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:00.583475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 430/874] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:00.597900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b643f0-9be0-424c-99d4-178ac298ba0d None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:00.598267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b643f0-9be0-424c-99d4-178ac298ba0d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:00.598576 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b643f0-9be0-424c-99d4-178ac298ba0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:00.617564 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-91b643f0-9be0-424c-99d4-178ac298ba0d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:00.618781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-91b643f0-9be0-424c-99d4-178ac298ba0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:00.699612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 445/875] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:00 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:00.722775 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb11ec-cb50-4d30-8b4e-36ab4c5d2840 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:00.795276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:00.829034 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bb11ec-cb50-4d30-8b4e-36ab4c5d2840 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:00.865854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80568055-c43e-41ca-889f-aecc75e8d1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:00.866750 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80568055-c43e-41ca-889f-aecc75e8d1dd None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:00.867183 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80568055-c43e-41ca-889f-aecc75e8d1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:00.867608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80568055-c43e-41ca-889f-aecc75e8d1dd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:00.882523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80568055-c43e-41ca-889f-aecc75e8d1dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:41:00.884301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-80568055-c43e-41ca-889f-aecc75e8d1dd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 12:41:00.884301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:00.884301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:00.884301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:00.884301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:00.884301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:00.884301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:00.884301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:00.884301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:00.884301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:00.884301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:00.884301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:00.884301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:41:00.884301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:41:00.884301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:41:00.884301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:41:00.885378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:41:00.885378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:41:00.885378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:41:00.885378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:41:00.885378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:00.885378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:00.885378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:00.885378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:00.885378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:00.885378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:00.885378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:00.885378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:00.885378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:00.885378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:00.885378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:00.885378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:00.886445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:00.886445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:00.886445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:00.886445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:00.886445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:41:00.886445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:41:00.886445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:41:00.886445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:41:00.886445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:41:00.886445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:41:00.886445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 12:41:00.886445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:00.886445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 431/876] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 3 12:41:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:00.893690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b643f0-9be0-424c-99d4-178ac298ba0d None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:00.941556 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:00.997792 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b643f0-9be0-424c-99d4-178ac298ba0d None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:01.000095 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fbe55-2a2b-4bca-a4e6-c15a43225a81 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:01.000386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fbe55-2a2b-4bca-a4e6-c15a43225a81 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:01.000666 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935fbe55-2a2b-4bca-a4e6-c15a43225a81 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:01.002144 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 432/877] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:01.007191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab7609-46d8-432d-a914-1590d8b2c77b None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:01.007473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab7609-46d8-432d-a914-1590d8b2c77b None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:01.007760 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab7609-46d8-432d-a914-1590d8b2c77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:01.027154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-edab7609-46d8-432d-a914-1590d8b2c77b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:01.028687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-edab7609-46d8-432d-a914-1590d8b2c77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:01.038677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc44177-8597-47c6-887d-d185c6b58486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:01.039692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc44177-8597-47c6-887d-d185c6b58486 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:01.040188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc44177-8597-47c6-887d-d185c6b58486 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:01.040657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc44177-8597-47c6-887d-d185c6b58486 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:01.041615 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc44177-8597-47c6-887d-d185c6b58486 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:01.042137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc44177-8597-47c6-887d-d185c6b58486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:01.043848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc44177-8597-47c6-887d-d185c6b58486 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:01.053721 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 446/878] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 3 12:41:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:01.059738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc44177-8597-47c6-887d-d185c6b58486 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:01.102650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 433/879] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:01 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:01.121684 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab7609-46d8-432d-a914-1590d8b2c77b None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:01.181760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc44177-8597-47c6-887d-d185c6b58486 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:01.194309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:01.195975 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:01.231017 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2008738-5e92-4773-9c4e-d06dcfa29ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:01.231658 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2008738-5e92-4773-9c4e-d06dcfa29ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:01.231940 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2008738-5e92-4773-9c4e-d06dcfa29ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:01.232217 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2008738-5e92-4773-9c4e-d06dcfa29ee2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:01.241193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab7609-46d8-432d-a914-1590d8b2c77b None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:01.245629 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2008738-5e92-4773-9c4e-d06dcfa29ee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:01.247141 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2008738-5e92-4773-9c4e-d06dcfa29ee2 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:01.262351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-e2008738-5e92-4773-9c4e-d06dcfa29ee2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:01.262351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:01.262351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:01.262351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:01.262351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:01.262351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:01.262351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:01.262351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:01.262351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:01.262351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:01.262351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:01.262351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:01.262351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:41:01.262351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:41:01.262351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:01.263317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:01.264231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:41:01.264231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:41:01.264231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:01.264231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:01.268174 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 447/880] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:01.269441 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2008738-5e92-4773-9c4e-d06dcfa29ee2 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:01.289377 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1e168-7d70-4a8d-a175-1d843d05fec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:01.290310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1e168-7d70-4a8d-a175-1d843d05fec9 None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:01.290828 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1e168-7d70-4a8d-a175-1d843d05fec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:01.291300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a1e168-7d70-4a8d-a175-1d843d05fec9 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:01.292703 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1e168-7d70-4a8d-a175-1d843d05fec9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:01.294855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a1e168-7d70-4a8d-a175-1d843d05fec9 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:01.319866 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 434/881] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 3 12:41:01 2022] POST /identity/v3/services => generated 269 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:41:01.388693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2008738-5e92-4773-9c4e-d06dcfa29ee2 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:01.439941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d0894-419d-4d09-b363-66c393015bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:01.441076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d0894-419d-4d09-b363-66c393015bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:01.441609 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d0894-419d-4d09-b363-66c393015bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:01.442083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d0894-419d-4d09-b363-66c393015bd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:01.443096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d0894-419d-4d09-b363-66c393015bd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:01.443657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d0894-419d-4d09-b363-66c393015bd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:01.445270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d0894-419d-4d09-b363-66c393015bd9 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:01.460038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 448/882] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:01.468203 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1e168-7d70-4a8d-a175-1d843d05fec9 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:01.580620 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a1e168-7d70-4a8d-a175-1d843d05fec9 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:01.603739 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:01.618864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae308ab8-9c16-4075-b574-5d5dfdb70c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:01.619524 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae308ab8-9c16-4075-b574-5d5dfdb70c74 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:01.619800 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae308ab8-9c16-4075-b574-5d5dfdb70c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:01.620076 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae308ab8-9c16-4075-b574-5d5dfdb70c74 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:01.628104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae308ab8-9c16-4075-b574-5d5dfdb70c74 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:01.629540 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae308ab8-9c16-4075-b574-5d5dfdb70c74 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:01.641146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-ae308ab8-9c16-4075-b574-5d5dfdb70c74 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 12:41:01.641146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:01.641146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:01.641146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:01.641146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:01.641146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:01.641146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:01.641146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:01.641146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:01.641146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:01.641146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:01.641146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:01.641146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:41:01.641146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:41:01.641146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:41:01.641146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:01.642087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:01.643101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:01.643101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:01.643101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:41:01.643101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:41:01.643101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:41:01.643101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:41:01.643101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:41:01.643101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:41:01.643101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 12:41:01.643101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:01.645601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 435/883] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 3 12:41:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:01.648420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d0894-419d-4d09-b363-66c393015bd9 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:01.669419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:01.760612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d0894-419d-4d09-b363-66c393015bd9 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:01.797806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f03daf-3cef-46cb-9449-df3a6e4e464a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:01.798711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f03daf-3cef-46cb-9449-df3a6e4e464a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:01.799170 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f03daf-3cef-46cb-9449-df3a6e4e464a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:01.799585 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f03daf-3cef-46cb-9449-df3a6e4e464a None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:01.800551 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f03daf-3cef-46cb-9449-df3a6e4e464a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:01.801000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f03daf-3cef-46cb-9449-df3a6e4e464a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:01.802570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f03daf-3cef-46cb-9449-df3a6e4e464a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:01.821127 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 449/884] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 3 12:41:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:01.826749 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae308ab8-9c16-4075-b574-5d5dfdb70c74 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:01.949003 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae308ab8-9c16-4075-b574-5d5dfdb70c74 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:01.984815 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e4b3c-5487-496b-a7a2-dec4ab7442bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:01.985684 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e4b3c-5487-496b-a7a2-dec4ab7442bd None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:01.986109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e4b3c-5487-496b-a7a2-dec4ab7442bd None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:01.986572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e4b3c-5487-496b-a7a2-dec4ab7442bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:02.022021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a982f7-84c5-4ba3-8b4d-89a4e4aa0341 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:02.023173 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a982f7-84c5-4ba3-8b4d-89a4e4aa0341 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:02.024726 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a982f7-84c5-4ba3-8b4d-89a4e4aa0341 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:02.026563 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 450/885] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:41:02.034330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebf2cf-5431-46c5-9407-59a89f73b3b4 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:02.034607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebf2cf-5431-46c5-9407-59a89f73b3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:02.034894 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ebf2cf-5431-46c5-9407-59a89f73b3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:02.042961 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:02.053723 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-39ebf2cf-5431-46c5-9407-59a89f73b3b4 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:02.054898 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-39ebf2cf-5431-46c5-9407-59a89f73b3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:02.070511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e4b3c-5487-496b-a7a2-dec4ab7442bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:02.072799 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e4b3c-5487-496b-a7a2-dec4ab7442bd None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:02.130852 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 451/886] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:02.149024 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebf2cf-5431-46c5-9407-59a89f73b3b4 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:02.152798 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:02.197671 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 436/887] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:41:01 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:41:02.203560 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e4b3c-5487-496b-a7a2-dec4ab7442bd None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:02.282843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ebf2cf-5431-46c5-9407-59a89f73b3b4 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:02.288975 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:02.309420 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e4b3c-5487-496b-a7a2-dec4ab7442bd None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:02.331411 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f219470c-2341-4ca8-8d0a-644b54c0875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:02.332055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219470c-2341-4ca8-8d0a-644b54c0875f None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:02.332330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219470c-2341-4ca8-8d0a-644b54c0875f None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:02.332601 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f219470c-2341-4ca8-8d0a-644b54c0875f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:02.333231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219470c-2341-4ca8-8d0a-644b54c0875f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:02.334636 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f219470c-2341-4ca8-8d0a-644b54c0875f None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-f219470c-2341-4ca8-8d0a-644b54c0875f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:02.343987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:02.344951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:02.344951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:02.344951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:02.344951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:02.344951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:02.344951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:02.344951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:41:02.344951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:41:02.344951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:41:02.344951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:41:02.344951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:02.344951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:02.344951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:02.344951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:02.344951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:02.344951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:02.345815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:02.345815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:02.345815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:02.345815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:02.345815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:02.345815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:02.345815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:02.345815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:02.345815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:02.345815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:02.345815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:41:02.345815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:02.345815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 12:41:02.345815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:02.345815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 452/888] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 3 12:41:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:02.347051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893fe31-fadc-45d5-91ec-c2c750ae6d8d None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:02.347393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893fe31-fadc-45d5-91ec-c2c750ae6d8d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:02.347583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4893fe31-fadc-45d5-91ec-c2c750ae6d8d None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:02.348937 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 453/889] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:02.352352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31aa25-0efb-4ed5-949e-33f0affeb748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:02.353097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31aa25-0efb-4ed5-949e-33f0affeb748 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:02.353354 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893fe31-fadc-45d5-91ec-c2c750ae6d8d None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:02.353452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31aa25-0efb-4ed5-949e-33f0affeb748 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:02.353707 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31aa25-0efb-4ed5-949e-33f0affeb748 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:02.365193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31aa25-0efb-4ed5-949e-33f0affeb748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:02.366721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31aa25-0efb-4ed5-949e-33f0affeb748 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:02.382455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-6f31aa25-0efb-4ed5-949e-33f0affeb748 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:02.382455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:02.382455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:02.382455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:02.382455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:02.382455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:02.382455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:02.382455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:02.382455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:02.382455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:02.382455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:02.382455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:02.382455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:41:02.382455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:41:02.382455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:02.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:02.384253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:41:02.384253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:41:02.384253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:02.384253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:02.384253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 437/890] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:02.385031 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5caf7-bb55-46f2-82a6-dc69f85ae23a None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:02.385143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5caf7-bb55-46f2-82a6-dc69f85ae23a None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:02.385358 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5caf7-bb55-46f2-82a6-dc69f85ae23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:02.404562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-97b5caf7-bb55-46f2-82a6-dc69f85ae23a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:02.405784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-97b5caf7-bb55-46f2-82a6-dc69f85ae23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:02.427994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:02.466132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4893fe31-fadc-45d5-91ec-c2c750ae6d8d None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:02.490506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 438/891] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:02.490506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5caf7-bb55-46f2-82a6-dc69f85ae23a None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:02.517985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad1e66-1610-421c-ac94-115af59aef64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:02.518785 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad1e66-1610-421c-ac94-115af59aef64 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:02.519129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad1e66-1610-421c-ac94-115af59aef64 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:02.519408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad1e66-1610-421c-ac94-115af59aef64 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:02.520077 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad1e66-1610-421c-ac94-115af59aef64 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:02.520376 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad1e66-1610-421c-ac94-115af59aef64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:02.521723 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad1e66-1610-421c-ac94-115af59aef64 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:02.527052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 454/892] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 3 12:41:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:02.527682 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad1e66-1610-421c-ac94-115af59aef64 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:02.553879 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:02.598893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5caf7-bb55-46f2-82a6-dc69f85ae23a None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:02.632547 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad1e66-1610-421c-ac94-115af59aef64 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:02.636925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ceb07-54cf-4c2c-920f-667cc8687666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:02.637535 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ceb07-54cf-4c2c-920f-667cc8687666 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:02.637819 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ceb07-54cf-4c2c-920f-667cc8687666 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:02.638092 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ceb07-54cf-4c2c-920f-667cc8687666 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:02.638822 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ceb07-54cf-4c2c-920f-667cc8687666 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:02.639130 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ceb07-54cf-4c2c-920f-667cc8687666 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:02.640671 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ceb07-54cf-4c2c-920f-667cc8687666 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:02.653054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 439/893] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:02.653677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ceb07-54cf-4c2c-920f-667cc8687666 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:02.665183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:02.671328 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446bc4b-70b3-43c8-a975-a41887d7b90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:02.672053 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446bc4b-70b3-43c8-a975-a41887d7b90c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:02.672428 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446bc4b-70b3-43c8-a975-a41887d7b90c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:02.672739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2446bc4b-70b3-43c8-a975-a41887d7b90c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:02.673605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446bc4b-70b3-43c8-a975-a41887d7b90c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:02.675261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2446bc4b-70b3-43c8-a975-a41887d7b90c None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-2446bc4b-70b3-43c8-a975-a41887d7b90c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:02.685033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:02.686016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:02.686016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:02.686016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:02.686016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:02.686016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:02.686016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:02.686016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:41:02.686016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:41:02.686016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:41:02.686016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:41:02.686016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:02.686016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:02.686016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:02.686016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:02.686016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:02.686016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:02.687612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:02.687612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:02.687612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:02.687612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:02.687612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:02.687612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:02.687612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:02.687612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:02.687612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:02.687612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:02.687612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:41:02.687612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:02.687612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 12:41:02.687612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:02.687612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 455/894] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:41:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:02.689048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446bc4b-70b3-43c8-a975-a41887d7b90c None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:02.761123 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ceb07-54cf-4c2c-920f-667cc8687666 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:02.783787 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:02.790782 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2446bc4b-70b3-43c8-a975-a41887d7b90c None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:02.796256 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cd30d-e02a-470e-baa9-be3bfc642dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:02.796870 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cd30d-e02a-470e-baa9-be3bfc642dfb None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:02.797143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cd30d-e02a-470e-baa9-be3bfc642dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:02.797410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842cd30d-e02a-470e-baa9-be3bfc642dfb None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:02.798078 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cd30d-e02a-470e-baa9-be3bfc642dfb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:02.798375 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cd30d-e02a-470e-baa9-be3bfc642dfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:02.799753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842cd30d-e02a-470e-baa9-be3bfc642dfb None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:02.814850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 440/895] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 3 12:41:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:02.815674 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cd30d-e02a-470e-baa9-be3bfc642dfb None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:02.831983 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4077d7-9c86-4675-87de-e96c90af9334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:02.832652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4077d7-9c86-4675-87de-e96c90af9334 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:02.832931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4077d7-9c86-4675-87de-e96c90af9334 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:02.833204 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4077d7-9c86-4675-87de-e96c90af9334 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:02.841718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4077d7-9c86-4675-87de-e96c90af9334 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:02.843400 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4077d7-9c86-4675-87de-e96c90af9334 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:02.856459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-2c4077d7-9c86-4675-87de-e96c90af9334 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:41:02.856459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:02.856459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:02.856459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:02.856459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:02.856459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:02.856459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:02.856459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:02.856459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:02.856459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:02.856459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:02.856459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:02.856459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:41:02.856459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:41:02.856459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:02.857447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:02.858927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:41:02.858927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:41:02.858927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:41:02.858927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:41:02.858927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:41:02.858927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:41:02.858927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:41:02.858927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:02.863041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 456/896] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:41:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:02.864601 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4077d7-9c86-4675-87de-e96c90af9334 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:02.939291 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:02.968247 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842cd30d-e02a-470e-baa9-be3bfc642dfb None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:03.002607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4077d7-9c86-4675-87de-e96c90af9334 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:03.009073 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ad9b3-4c45-48d1-bf98-2ac31e3ded01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:03.009778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ad9b3-4c45-48d1-bf98-2ac31e3ded01 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:03.010133 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ad9b3-4c45-48d1-bf98-2ac31e3ded01 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:03.010543 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ad9b3-4c45-48d1-bf98-2ac31e3ded01 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:03.011428 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ad9b3-4c45-48d1-bf98-2ac31e3ded01 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:03.011753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ad9b3-4c45-48d1-bf98-2ac31e3ded01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:03.013421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ad9b3-4c45-48d1-bf98-2ac31e3ded01 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:03.018647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 441/897] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:41:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:03.023840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ad9b3-4c45-48d1-bf98-2ac31e3ded01 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:03.047424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc652118-02ea-482d-86cb-71257db6de7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:03.048887 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc652118-02ea-482d-86cb-71257db6de7a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:03.049344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc652118-02ea-482d-86cb-71257db6de7a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:03.050186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc652118-02ea-482d-86cb-71257db6de7a None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:03.051136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc652118-02ea-482d-86cb-71257db6de7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:03.051594 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc652118-02ea-482d-86cb-71257db6de7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:03.053819 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc652118-02ea-482d-86cb-71257db6de7a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:03.071662 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 457/898] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:41:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:03.077110 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc652118-02ea-482d-86cb-71257db6de7a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:03.091964 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:03.141964 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ad9b3-4c45-48d1-bf98-2ac31e3ded01 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:03.179921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d365c82f-9b08-43d9-9f22-12b6087a4f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:03.180658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365c82f-9b08-43d9-9f22-12b6087a4f52 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:03.181025 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365c82f-9b08-43d9-9f22-12b6087a4f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:03.181374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d365c82f-9b08-43d9-9f22-12b6087a4f52 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:03.182134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d365c82f-9b08-43d9-9f22-12b6087a4f52 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:03.182523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d365c82f-9b08-43d9-9f22-12b6087a4f52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:03.183984 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d365c82f-9b08-43d9-9f22-12b6087a4f52 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:03.194359 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 442/899] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:41:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:03.195446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d365c82f-9b08-43d9-9f22-12b6087a4f52 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:03.201363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc652118-02ea-482d-86cb-71257db6de7a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:03.219980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:03.241437 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8378dc-48f1-43bb-a32e-2a2669f968ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:03.242594 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8378dc-48f1-43bb-a32e-2a2669f968ab None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:03.243058 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8378dc-48f1-43bb-a32e-2a2669f968ab None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:03.243487 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8378dc-48f1-43bb-a32e-2a2669f968ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:03.244414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8378dc-48f1-43bb-a32e-2a2669f968ab None admin] RBAC: Adding query filter params (role.id=b9b9695215b144d4b974ba1266bb18c1, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=3ae6eeb3d4f94925a5e1dd497d352d2e) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:03.244963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8378dc-48f1-43bb-a32e-2a2669f968ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:03.246700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8378dc-48f1-43bb-a32e-2a2669f968ab None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:03.258258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 458/900] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:41:03 2022] GET /identity/v3/role_assignments?user.id=3ae6eeb3d4f94925a5e1dd497d352d2e&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=b9b9695215b144d4b974ba1266bb18c1 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:03.311942 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d365c82f-9b08-43d9-9f22-12b6087a4f52 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:03.331025 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:03.351623 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588efd9-56b1-4f17-a94f-0059441bbbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:03.352263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588efd9-56b1-4f17-a94f-0059441bbbcf None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:03.352546 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588efd9-56b1-4f17-a94f-0059441bbbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:03.352826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588efd9-56b1-4f17-a94f-0059441bbbcf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:03.353604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588efd9-56b1-4f17-a94f-0059441bbbcf None admin] RBAC: Adding query filter params (interface=public, service_id=f749d9f9b8ed482bbe6de0ff68b127fd, region_id=RegionOne) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:03.353916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588efd9-56b1-4f17-a94f-0059441bbbcf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:03.355356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588efd9-56b1-4f17-a94f-0059441bbbcf None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:03.361790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 443/901] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 3 12:41:03 2022] GET /identity/v3/endpoints?service_id=f749d9f9b8ed482bbe6de0ff68b127fd&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:03.367423 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8378dc-48f1-43bb-a32e-2a2669f968ab None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:03.476242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8378dc-48f1-43bb-a32e-2a2669f968ab None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:03.505681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35fabd4-ce3a-48fc-9590-0ed427c22574 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:03.506003 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35fabd4-ce3a-48fc-9590-0ed427c22574 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:03.506276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35fabd4-ce3a-48fc-9590-0ed427c22574 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:03.507876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 444/902] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:03.512644 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6888d7f-cd1e-45bd-8906-55d22c2a7834 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:03.512932 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6888d7f-cd1e-45bd-8906-55d22c2a7834 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:03.513213 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6888d7f-cd1e-45bd-8906-55d22c2a7834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:03.515246 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4235856-94c6-4317-8a74-f7c0fb9fd58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:03.516113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4235856-94c6-4317-8a74-f7c0fb9fd58d None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:03.516588 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4235856-94c6-4317-8a74-f7c0fb9fd58d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:03.517137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4235856-94c6-4317-8a74-f7c0fb9fd58d None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:03.518072 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4235856-94c6-4317-8a74-f7c0fb9fd58d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:03.519815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4235856-94c6-4317-8a74-f7c0fb9fd58d None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:03.526360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 459/903] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 3 12:41:03 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:41:03.533580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-e6888d7f-cd1e-45bd-8906-55d22c2a7834 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:03.534840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e6888d7f-cd1e-45bd-8906-55d22c2a7834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:03.616082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 445/904] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 12:41:03.637689 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4235856-94c6-4317-8a74-f7c0fb9fd58d None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:03.693953 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:03.756490 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4235856-94c6-4317-8a74-f7c0fb9fd58d None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:03.792505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8c64c-91b7-4000-9290-919022944275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:03.793442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8c64c-91b7-4000-9290-919022944275 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:03.793931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8c64c-91b7-4000-9290-919022944275 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:03.794383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8c64c-91b7-4000-9290-919022944275 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:03.795259 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8c64c-91b7-4000-9290-919022944275 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:03.796832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d8c64c-91b7-4000-9290-919022944275 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:03.806212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-55d8c64c-91b7-4000-9290-919022944275 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 12:41:03.806212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:03.806212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:41:03.806212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:41:03.806212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:41:03.806212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:41:03.806212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:41:03.806212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:03.806212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 12:41:03.806212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:03.806212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:03.806212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:03.806212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:03.806212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:03.806212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:03.806212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:03.807194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:03.807194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:03.807194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:03.807194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:03.807194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:03.807194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:03.807194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:03.807194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:41:03.807194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:41:03.807194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:41:03.807194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:41:03.807194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:03.807194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:03.807194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:03.807194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:03.807194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:03.808056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:03.808056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:03.808056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:03.808056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:03.808056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:03.808056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:03.808056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:03.808056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:03.808056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:03.808056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:03.808056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:03.808056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:41:03.808056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:03.808056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 12:41:03.808056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:03.810940 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 460/905] 10.222.0.65 () {56 vars in 1132 bytes} [Wed Aug 3 12:41:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:03.817628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6888d7f-cd1e-45bd-8906-55d22c2a7834 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:03.926199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6888d7f-cd1e-45bd-8906-55d22c2a7834 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:03.938321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:03.972318 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c44bb3-a12c-4d66-a1e2-d300577a567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:03.972963 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c44bb3-a12c-4d66-a1e2-d300577a567b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:03.973245 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c44bb3-a12c-4d66-a1e2-d300577a567b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:03.973518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c44bb3-a12c-4d66-a1e2-d300577a567b None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:03.974223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c44bb3-a12c-4d66-a1e2-d300577a567b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:03.974552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c44bb3-a12c-4d66-a1e2-d300577a567b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:03.975946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c44bb3-a12c-4d66-a1e2-d300577a567b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:03.981747 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 446/906] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 3 12:41:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:03.987225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8c64c-91b7-4000-9290-919022944275 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:04.096657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d8c64c-91b7-4000-9290-919022944275 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:04.106246 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:04.133189 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fe2a3-f5ca-4907-9269-713c3f7b6ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:04.133745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fe2a3-f5ca-4907-9269-713c3f7b6ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:04.134029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fe2a3-f5ca-4907-9269-713c3f7b6ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:04.134309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6fe2a3-f5ca-4907-9269-713c3f7b6ed7 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:04.135029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6fe2a3-f5ca-4907-9269-713c3f7b6ed7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:04.135338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6fe2a3-f5ca-4907-9269-713c3f7b6ed7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:04.136713 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6fe2a3-f5ca-4907-9269-713c3f7b6ed7 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:04.142098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 461/907] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 3 12:41:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:04.147892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c44bb3-a12c-4d66-a1e2-d300577a567b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:04.258355 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c44bb3-a12c-4d66-a1e2-d300577a567b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:04.292966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fca38b-5e96-4cdd-998d-6c27eed862f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:04.293584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fca38b-5e96-4cdd-998d-6c27eed862f6 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:04.293866 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fca38b-5e96-4cdd-998d-6c27eed862f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:04.294142 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fca38b-5e96-4cdd-998d-6c27eed862f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:04.295401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fca38b-5e96-4cdd-998d-6c27eed862f6 None admin] RBAC: Adding query filter params (interface=public, service_id=a27aae1231104135ae58910d9ab403f8, region_id=RegionOne) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:04.296242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fca38b-5e96-4cdd-998d-6c27eed862f6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:04.298543 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fca38b-5e96-4cdd-998d-6c27eed862f6 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:04.305138 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 447/908] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 3 12:41:04 2022] GET /identity/v3/endpoints?service_id=a27aae1231104135ae58910d9ab403f8&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:04.310934 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fe2a3-f5ca-4907-9269-713c3f7b6ed7 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:04.416644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6fe2a3-f5ca-4907-9269-713c3f7b6ed7 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:04.456316 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ce81d-01ae-4c0b-a0ad-0d0e245ef66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:04.456939 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ce81d-01ae-4c0b-a0ad-0d0e245ef66e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:04.457224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ce81d-01ae-4c0b-a0ad-0d0e245ef66e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:04.457502 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ce81d-01ae-4c0b-a0ad-0d0e245ef66e None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:04.458164 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ce81d-01ae-4c0b-a0ad-0d0e245ef66e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:04.459570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ce81d-01ae-4c0b-a0ad-0d0e245ef66e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:04.465700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 462/909] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 3 12:41:04 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:41:04.940634 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b380f-fcf1-4716-9b19-6ecce5b4ac9e None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:04.941165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b380f-fcf1-4716-9b19-6ecce5b4ac9e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:04.941165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b380f-fcf1-4716-9b19-6ecce5b4ac9e None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:04.942822 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 448/910] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:04.948067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b2122-da62-42da-a27a-9a6b8493e006 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:04.948364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b2122-da62-42da-a27a-9a6b8493e006 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:04.948644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b2122-da62-42da-a27a-9a6b8493e006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:04.967328 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-a54b2122-da62-42da-a27a-9a6b8493e006 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:04.969081 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a54b2122-da62-42da-a27a-9a6b8493e006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:04.998741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:05.042595 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 463/911] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:05.061557 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b380f-fcf1-4716-9b19-6ecce5b4ac9e None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:05.106305 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:05.175131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b380f-fcf1-4716-9b19-6ecce5b4ac9e None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:05.211334 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e2fe3-70e7-4807-a680-1b06b6d1132a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:05.211957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e2fe3-70e7-4807-a680-1b06b6d1132a None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:05.212236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e2fe3-70e7-4807-a680-1b06b6d1132a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:05.212511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e2fe3-70e7-4807-a680-1b06b6d1132a None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:05.213209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e2fe3-70e7-4807-a680-1b06b6d1132a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:05.214605 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e2fe3-70e7-4807-a680-1b06b6d1132a None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:05.233013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 449/912] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 3 12:41:05 2022] POST /identity/v3/services => generated 241 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:41:05.308086 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62400da-9884-4d66-9931-7b81b737cbc3 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:05.308618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62400da-9884-4d66-9931-7b81b737cbc3 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:05.308618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62400da-9884-4d66-9931-7b81b737cbc3 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:05.310587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 464/913] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:05.316776 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb2c0f-d7a2-4d4b-9e95-865476b14152 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:05.317099 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb2c0f-d7a2-4d4b-9e95-865476b14152 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:05.317327 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb2c0f-d7a2-4d4b-9e95-865476b14152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:05.337473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-39cb2c0f-d7a2-4d4b-9e95-865476b14152 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:05.338843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-39cb2c0f-d7a2-4d4b-9e95-865476b14152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:05.421418 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 450/914] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:05.443764 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62400da-9884-4d66-9931-7b81b737cbc3 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:05.556021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62400da-9884-4d66-9931-7b81b737cbc3 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:05.594003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539af4f-83aa-482f-9ba6-09d3b380b301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:05.594701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539af4f-83aa-482f-9ba6-09d3b380b301 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:05.594985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539af4f-83aa-482f-9ba6-09d3b380b301 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:05.595261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539af4f-83aa-482f-9ba6-09d3b380b301 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:05.607961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539af4f-83aa-482f-9ba6-09d3b380b301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:41:05.613573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-a539af4f-83aa-482f-9ba6-09d3b380b301 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 12:41:05.613573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:05.613573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:05.613573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:05.613573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:05.613573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:05.613573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:05.613573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:05.613573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:05.613573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:05.613573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:05.613573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:05.613573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:41:05.613573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:41:05.613573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:41:05.613573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:41:05.614526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:41:05.614526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:41:05.614526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:41:05.614526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:41:05.614526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:05.614526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:05.614526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:05.614526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:05.614526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:05.614526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:05.614526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:05.614526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:05.614526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:05.614526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:05.614526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:05.614526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:05.615373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:05.615373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:05.615373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:05.615373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:05.615373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:41:05.615373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:41:05.615373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:41:05.615373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:41:05.615373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:41:05.615373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:41:05.615373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 12:41:05.615373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:05.615373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 465/915] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 3 12:41:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:05.620782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb2c0f-d7a2-4d4b-9e95-865476b14152 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:05.647544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:05.672288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab43ba-53b8-4dc9-8c81-597bf471032f None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:05.672569 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab43ba-53b8-4dc9-8c81-597bf471032f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:05.672850 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab43ba-53b8-4dc9-8c81-597bf471032f None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:05.674448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 466/916] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:41:05.679195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed46c49-b4b3-4288-aed7-25a4d0d5facd None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:05.679484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed46c49-b4b3-4288-aed7-25a4d0d5facd None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:05.679764 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed46c49-b4b3-4288-aed7-25a4d0d5facd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:05.702911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1ed46c49-b4b3-4288-aed7-25a4d0d5facd None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:05.704166 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1ed46c49-b4b3-4288-aed7-25a4d0d5facd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:05.728721 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:05.734154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb2c0f-d7a2-4d4b-9e95-865476b14152 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:05.771733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63458ba-009d-46c9-a43b-7f7435e5b03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:05.772356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63458ba-009d-46c9-a43b-7f7435e5b03a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:05.772636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63458ba-009d-46c9-a43b-7f7435e5b03a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:05.772908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63458ba-009d-46c9-a43b-7f7435e5b03a None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:05.773681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63458ba-009d-46c9-a43b-7f7435e5b03a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:05.773991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63458ba-009d-46c9-a43b-7f7435e5b03a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:05.775553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63458ba-009d-46c9-a43b-7f7435e5b03a None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:05.786262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 451/917] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 3 12:41:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:05.789734 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 467/918] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 12:41:05.791407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63458ba-009d-46c9-a43b-7f7435e5b03a None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:05.813174 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed46c49-b4b3-4288-aed7-25a4d0d5facd None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:05.909006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:05.925628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63458ba-009d-46c9-a43b-7f7435e5b03a None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:05.947049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed46c49-b4b3-4288-aed7-25a4d0d5facd None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:05.968161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d8668-53a7-442d-82a1-aefb226ada1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:05.969569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d8668-53a7-442d-82a1-aefb226ada1b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:05.970055 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d8668-53a7-442d-82a1-aefb226ada1b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:05.970926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d8668-53a7-442d-82a1-aefb226ada1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:05.984572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d8668-53a7-442d-82a1-aefb226ada1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:05.985136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a643fc7-0992-4e57-93f9-09c59f9b3f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:05.985857 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a643fc7-0992-4e57-93f9-09c59f9b3f4b None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:05.986211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a643fc7-0992-4e57-93f9-09c59f9b3f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:05.986684 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a643fc7-0992-4e57-93f9-09c59f9b3f4b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:05.987126 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d8668-53a7-442d-82a1-aefb226ada1b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:05.987537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a643fc7-0992-4e57-93f9-09c59f9b3f4b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:05.988997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a643fc7-0992-4e57-93f9-09c59f9b3f4b None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-0a643fc7-0992-4e57-93f9-09c59f9b3f4b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:05.998312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:05.999862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:05.999862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:05.999862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:05.999862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:05.999862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:05.999862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:05.999862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:41:05.999862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:41:05.999862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:41:05.999862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:41:05.999862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:05.999862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:05.999862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:05.999862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:05.999862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:05.999862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:06.000720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:06.000720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:06.000720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:06.000720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:06.000720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:06.000720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:06.000720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:06.000720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:06.000720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:06.000720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:06.000720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:41:06.000720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:06.000720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 03 12:41:06.000720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:06.005149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 468/919] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:41:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:06.006057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a643fc7-0992-4e57-93f9-09c59f9b3f4b None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:06.007316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-ee6d8668-53a7-442d-82a1-aefb226ada1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:06.007316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:06.007316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:06.007316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:06.007316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:06.007316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:06.007316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:06.007316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:06.007316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:06.007316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:06.007316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:06.007316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:06.007316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:41:06.007316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:41:06.007316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:06.008184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:06.009145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:41:06.009145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:41:06.009145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:06.009145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:06.012172 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 452/920] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:06.020664 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d8668-53a7-442d-82a1-aefb226ada1b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:06.033936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:06.116915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a643fc7-0992-4e57-93f9-09c59f9b3f4b None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:06.133746 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d8668-53a7-442d-82a1-aefb226ada1b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:06.153072 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0ce7e-92bd-4cfd-8ecb-19aa237d0dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:06.153843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0ce7e-92bd-4cfd-8ecb-19aa237d0dad None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:06.154227 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0ce7e-92bd-4cfd-8ecb-19aa237d0dad None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:06.154634 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0ce7e-92bd-4cfd-8ecb-19aa237d0dad None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:06.155446 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0ce7e-92bd-4cfd-8ecb-19aa237d0dad None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:06.156071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0ce7e-92bd-4cfd-8ecb-19aa237d0dad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:06.157302 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0ce7e-92bd-4cfd-8ecb-19aa237d0dad None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:06.166823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 469/921] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:41:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:06.168269 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0ce7e-92bd-4cfd-8ecb-19aa237d0dad None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:06.170859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12928d9d-9624-43e4-b2ce-6ddb938b3488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:06.171741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12928d9d-9624-43e4-b2ce-6ddb938b3488 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:06.172164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12928d9d-9624-43e4-b2ce-6ddb938b3488 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:06.172607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12928d9d-9624-43e4-b2ce-6ddb938b3488 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:06.173474 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12928d9d-9624-43e4-b2ce-6ddb938b3488 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:06.173968 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12928d9d-9624-43e4-b2ce-6ddb938b3488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:06.175528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12928d9d-9624-43e4-b2ce-6ddb938b3488 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:06.182858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:06.194837 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 453/922] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:06.196085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12928d9d-9624-43e4-b2ce-6ddb938b3488 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:06.261748 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:06.280919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0ce7e-92bd-4cfd-8ecb-19aa237d0dad None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:06.300656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12928d9d-9624-43e4-b2ce-6ddb938b3488 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:06.318238 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f7d52-7f33-4321-b808-c0d2cf4d3d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:06.318905 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f7d52-7f33-4321-b808-c0d2cf4d3d9f None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:06.319184 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f7d52-7f33-4321-b808-c0d2cf4d3d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:06.319456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f7d52-7f33-4321-b808-c0d2cf4d3d9f None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:06.320237 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f7d52-7f33-4321-b808-c0d2cf4d3d9f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:06.320600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f7d52-7f33-4321-b808-c0d2cf4d3d9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:06.322054 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f7d52-7f33-4321-b808-c0d2cf4d3d9f None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:06.332619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 470/923] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 3 12:41:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:06.333465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f7d52-7f33-4321-b808-c0d2cf4d3d9f None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:06.339958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe676f8e-4728-4fd0-a212-920f1d054946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:06.340954 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe676f8e-4728-4fd0-a212-920f1d054946 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:06.341430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe676f8e-4728-4fd0-a212-920f1d054946 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:06.341928 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe676f8e-4728-4fd0-a212-920f1d054946 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:06.349961 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe676f8e-4728-4fd0-a212-920f1d054946 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:06.351778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe676f8e-4728-4fd0-a212-920f1d054946 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:06.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-fe676f8e-4728-4fd0-a212-920f1d054946 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 12:41:06.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:06.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:06.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:06.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:06.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:06.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:06.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:06.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:06.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:06.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:06.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:06.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:41:06.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:41:06.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:41:06.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:06.365553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:06.366527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:06.366527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:06.366527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:41:06.366527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:41:06.366527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:41:06.366527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:41:06.366527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:41:06.366527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:41:06.366527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 12:41:06.366527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:06.368839 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 454/924] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 3 12:41:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:06.376393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe676f8e-4728-4fd0-a212-920f1d054946 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:06.442657 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:06.464576 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f7d52-7f33-4321-b808-c0d2cf4d3d9f None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:06.500697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c5a2a-40d5-49a2-aa0f-1e71e7cc871a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:06.501372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c5a2a-40d5-49a2-aa0f-1e71e7cc871a None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:06.501659 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c5a2a-40d5-49a2-aa0f-1e71e7cc871a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:06.501936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c5a2a-40d5-49a2-aa0f-1e71e7cc871a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:06.502706 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c5a2a-40d5-49a2-aa0f-1e71e7cc871a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:06.503386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe676f8e-4728-4fd0-a212-920f1d054946 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:06.504071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c5a2a-40d5-49a2-aa0f-1e71e7cc871a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:06.533289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 471/925] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:41:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:41:06.533289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b2384-3e8a-448c-b6c7-6a4cbde406b7 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:06.533289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b2384-3e8a-448c-b6c7-6a4cbde406b7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:06.533289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b2384-3e8a-448c-b6c7-6a4cbde406b7 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:06.533289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 472/926] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:06.534981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014ce69-0be0-4ec4-a356-0ca1bbb624e0 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:06.535276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014ce69-0be0-4ec4-a356-0ca1bbb624e0 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:06.535558 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014ce69-0be0-4ec4-a356-0ca1bbb624e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:06.543927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:06.558183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1014ce69-0be0-4ec4-a356-0ca1bbb624e0 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:06.558757 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51235d64-9d37-4a7c-9cff-250fe6e8cea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:06.559849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51235d64-9d37-4a7c-9cff-250fe6e8cea9 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:06.560048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1014ce69-0be0-4ec4-a356-0ca1bbb624e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:06.560431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51235d64-9d37-4a7c-9cff-250fe6e8cea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:06.560848 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51235d64-9d37-4a7c-9cff-250fe6e8cea9 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:06.562581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51235d64-9d37-4a7c-9cff-250fe6e8cea9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:06.563079 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51235d64-9d37-4a7c-9cff-250fe6e8cea9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:06.564797 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51235d64-9d37-4a7c-9cff-250fe6e8cea9 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:06.587568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 455/927] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 3 12:41:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:06.598526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51235d64-9d37-4a7c-9cff-250fe6e8cea9 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:06.644897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 473/928] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 12:41:06.667400 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014ce69-0be0-4ec4-a356-0ca1bbb624e0 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:06.724805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51235d64-9d37-4a7c-9cff-250fe6e8cea9 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:06.741098 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:06.770715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75477f60-8f00-47cb-a2a7-6fbe65072318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:06.771620 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75477f60-8f00-47cb-a2a7-6fbe65072318 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:06.771900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75477f60-8f00-47cb-a2a7-6fbe65072318 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:06.772175 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75477f60-8f00-47cb-a2a7-6fbe65072318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:06.787484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75477f60-8f00-47cb-a2a7-6fbe65072318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:06.789706 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75477f60-8f00-47cb-a2a7-6fbe65072318 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:06.796346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014ce69-0be0-4ec4-a356-0ca1bbb624e0 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:06.808647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-75477f60-8f00-47cb-a2a7-6fbe65072318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:06.808647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:06.808647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:06.808647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:06.808647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:06.808647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:06.808647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:06.808647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:06.808647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:06.808647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:06.808647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:06.808647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:06.808647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:41:06.808647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:41:06.808647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:06.809829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:06.810773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:41:06.810773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:41:06.810773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:06.810773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:06.812545 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 456/929] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:06.817581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75477f60-8f00-47cb-a2a7-6fbe65072318 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:06.849984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d01d7-8818-40c2-9dcd-32096795100d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:06.851129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d01d7-8818-40c2-9dcd-32096795100d None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:06.851513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d01d7-8818-40c2-9dcd-32096795100d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:06.852001 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d01d7-8818-40c2-9dcd-32096795100d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:06.852897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d01d7-8818-40c2-9dcd-32096795100d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:06.854861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d01d7-8818-40c2-9dcd-32096795100d None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:06.859880 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:06.873052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-929d01d7-8818-40c2-9dcd-32096795100d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 12:41:06.873052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:06.873052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:41:06.873052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:41:06.873052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:41:06.873052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:41:06.873052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:41:06.873052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:06.873052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 12:41:06.873052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:06.873052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:06.873052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:06.873052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:06.873052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:06.873052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:06.873052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:06.874096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:06.874096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:06.874096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:06.874096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:06.874096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:06.874096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:06.874096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:06.874096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:41:06.874096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:41:06.874096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:41:06.874096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:41:06.874096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:06.874096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:06.874096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:06.874096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:06.874096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:06.874991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:06.874991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:06.874991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:06.874991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:06.874991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:06.874991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:06.874991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:06.874991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:06.874991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:06.874991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:06.874991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:06.874991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:41:06.874991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:06.874991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 03 12:41:06.874991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:06.879334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 474/930] 10.222.0.65 () {56 vars in 1132 bytes} [Wed Aug 3 12:41:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:06.885005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d01d7-8818-40c2-9dcd-32096795100d None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:06.964466 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75477f60-8f00-47cb-a2a7-6fbe65072318 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:07.011676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ba306-1174-45d4-882d-97a686f08e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:07.012457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ba306-1174-45d4-882d-97a686f08e6a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:07.012848 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ba306-1174-45d4-882d-97a686f08e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:07.013216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ba306-1174-45d4-882d-97a686f08e6a None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:07.014111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ba306-1174-45d4-882d-97a686f08e6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:07.014758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ba306-1174-45d4-882d-97a686f08e6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:07.016837 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ba306-1174-45d4-882d-97a686f08e6a None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:07.017767 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d01d7-8818-40c2-9dcd-32096795100d None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:07.036123 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 457/931] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:07.037946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ba306-1174-45d4-882d-97a686f08e6a None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:07.065795 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c913484-0c1e-4d85-a826-ccfd59ba6172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:07.066417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c913484-0c1e-4d85-a826-ccfd59ba6172 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:07.066726 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c913484-0c1e-4d85-a826-ccfd59ba6172 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:07.066995 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c913484-0c1e-4d85-a826-ccfd59ba6172 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:07.067671 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c913484-0c1e-4d85-a826-ccfd59ba6172 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:07.067977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c913484-0c1e-4d85-a826-ccfd59ba6172 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:07.069887 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c913484-0c1e-4d85-a826-ccfd59ba6172 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:07.074987 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:07.080289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 475/932] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 3 12:41:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:07.084886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c913484-0c1e-4d85-a826-ccfd59ba6172 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:07.165241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:07.182622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ba306-1174-45d4-882d-97a686f08e6a None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:07.221974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c913484-0c1e-4d85-a826-ccfd59ba6172 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:07.226518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df42a8b-de4f-42c6-a7b7-76ce8cf0c781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:07.227401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df42a8b-de4f-42c6-a7b7-76ce8cf0c781 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:07.227879 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df42a8b-de4f-42c6-a7b7-76ce8cf0c781 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:07.228351 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df42a8b-de4f-42c6-a7b7-76ce8cf0c781 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:07.237232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df42a8b-de4f-42c6-a7b7-76ce8cf0c781 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:07.239586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df42a8b-de4f-42c6-a7b7-76ce8cf0c781 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:07.254957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-6df42a8b-de4f-42c6-a7b7-76ce8cf0c781 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:41:07.254957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:07.254957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:07.254957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:07.254957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:07.254957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:07.254957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:07.254957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:07.254957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:07.254957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:07.254957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:07.254957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:07.254957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:41:07.254957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:41:07.254957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:07.255886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:07.256786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:41:07.256786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:41:07.256786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:41:07.256786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:41:07.256786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:41:07.256786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:41:07.256786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:41:07.256786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:07.259488 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 458/933] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:41:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:07.262011 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54cd48-a3e2-49fb-9e9a-8d4a79b997b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:07.266626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54cd48-a3e2-49fb-9e9a-8d4a79b997b6 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:07.266626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54cd48-a3e2-49fb-9e9a-8d4a79b997b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:07.266626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54cd48-a3e2-49fb-9e9a-8d4a79b997b6 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:07.266626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54cd48-a3e2-49fb-9e9a-8d4a79b997b6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:07.266626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54cd48-a3e2-49fb-9e9a-8d4a79b997b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:07.267021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df42a8b-de4f-42c6-a7b7-76ce8cf0c781 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:07.267106 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54cd48-a3e2-49fb-9e9a-8d4a79b997b6 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:07.278561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 476/934] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 3 12:41:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:07.281982 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54cd48-a3e2-49fb-9e9a-8d4a79b997b6 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:07.375832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:07.393273 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df42a8b-de4f-42c6-a7b7-76ce8cf0c781 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:07.408657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54cd48-a3e2-49fb-9e9a-8d4a79b997b6 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:07.428991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d7dbd-27f7-48ed-a9de-5a46b8d06dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:07.429613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d7dbd-27f7-48ed-a9de-5a46b8d06dce None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:07.429895 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d7dbd-27f7-48ed-a9de-5a46b8d06dce None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:07.430163 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d7dbd-27f7-48ed-a9de-5a46b8d06dce None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:07.430957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d7dbd-27f7-48ed-a9de-5a46b8d06dce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:07.431247 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d7dbd-27f7-48ed-a9de-5a46b8d06dce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:07.432669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d7dbd-27f7-48ed-a9de-5a46b8d06dce None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:07.436920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:07.449820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 459/935] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:41:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:07.449820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3478-8557-494c-9033-13ae66929e33 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:07.449820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3478-8557-494c-9033-13ae66929e33 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:07.449820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec3478-8557-494c-9033-13ae66929e33 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:07.449820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 460/936] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:07.451450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec3478-8557-494c-9033-13ae66929e33 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:07.454403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4000b92-b621-46ee-b95d-31d1ae65811d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:07.455057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4000b92-b621-46ee-b95d-31d1ae65811d None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:07.455339 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4000b92-b621-46ee-b95d-31d1ae65811d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:07.455611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4000b92-b621-46ee-b95d-31d1ae65811d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:07.456319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4000b92-b621-46ee-b95d-31d1ae65811d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:07.457676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4000b92-b621-46ee-b95d-31d1ae65811d None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:07.480692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 477/937] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:41:07 2022] POST /identity/v3/endpoints => generated 347 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:41:07.488038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c19d8f-cc6a-47c5-8a0c-4db49bde5e45 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:07.488355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c19d8f-cc6a-47c5-8a0c-4db49bde5e45 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:07.488591 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c19d8f-cc6a-47c5-8a0c-4db49bde5e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:07.509338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-e2c19d8f-cc6a-47c5-8a0c-4db49bde5e45 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:07.510353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e2c19d8f-cc6a-47c5-8a0c-4db49bde5e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:07.602083 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec3478-8557-494c-9033-13ae66929e33 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:07.602752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 478/938] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:07.617692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c19d8f-cc6a-47c5-8a0c-4db49bde5e45 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:07.655652 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f9645-7cbe-4235-8cb6-87175d07626b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:07.656222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f9645-7cbe-4235-8cb6-87175d07626b None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:07.656503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f9645-7cbe-4235-8cb6-87175d07626b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:07.656777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f9645-7cbe-4235-8cb6-87175d07626b None admin] PATH_INFO: `/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/3ae6eeb3d4f94925a5e1dd497d352d2e/roles/b9b9695215b144d4b974ba1266bb18c1` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:07.659394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f9645-7cbe-4235-8cb6-87175d07626b None admin] RBAC: Authorizing `identity:create_grant(project_id=c0774035ad0f41c986771ef4a41c7d60, user_id=3ae6eeb3d4f94925a5e1dd497d352d2e, role_id=b9b9695215b144d4b974ba1266bb18c1)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:07.660882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f9645-7cbe-4235-8cb6-87175d07626b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:07.679227 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 461/939] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:41:07 2022] PUT /identity/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/3ae6eeb3d4f94925a5e1dd497d352d2e/roles/b9b9695215b144d4b974ba1266bb18c1 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:41:07.742376 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c19d8f-cc6a-47c5-8a0c-4db49bde5e45 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:07.754003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:07.785758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e53fa3-7235-4a10-aa63-e2b1e9937032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:07.786744 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e53fa3-7235-4a10-aa63-e2b1e9937032 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:07.787265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e53fa3-7235-4a10-aa63-e2b1e9937032 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:07.787735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e53fa3-7235-4a10-aa63-e2b1e9937032 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:07.788561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e53fa3-7235-4a10-aa63-e2b1e9937032 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:07.790109 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e53fa3-7235-4a10-aa63-e2b1e9937032 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-98e53fa3-7235-4a10-aa63-e2b1e9937032 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:07.800697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:07.802007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:07.802007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:07.802007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:07.802007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:07.802007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:07.802007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:07.802007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:41:07.802007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:41:07.802007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:41:07.802007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:41:07.802007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:07.802007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:07.802007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:07.802007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:07.802007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:07.802007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:07.803102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:07.803102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:07.803102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:07.803102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:07.803102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:07.803102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:07.803102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:07.803102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:07.803102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:07.803102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:07.803102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:41:07.803102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:07.803102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 12:41:07.803102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:07.805726 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 479/940] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 3 12:41:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:07.815660 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f9645-7cbe-4235-8cb6-87175d07626b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:07.828706 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:07.926380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f9645-7cbe-4235-8cb6-87175d07626b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:07.961968 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1187d40-b62b-4488-ac0d-b4cdaf909881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:07.962629 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1187d40-b62b-4488-ac0d-b4cdaf909881 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:07.962912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1187d40-b62b-4488-ac0d-b4cdaf909881 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:07.963184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1187d40-b62b-4488-ac0d-b4cdaf909881 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:07.963932 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1187d40-b62b-4488-ac0d-b4cdaf909881 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:07.964236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1187d40-b62b-4488-ac0d-b4cdaf909881 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:07.965825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1187d40-b62b-4488-ac0d-b4cdaf909881 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:07.970923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 462/941] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 3 12:41:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:07.976659 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e53fa3-7235-4a10-aa63-e2b1e9937032 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:08.086460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e53fa3-7235-4a10-aa63-e2b1e9937032 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:08.122838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9df59-9111-473e-8b94-99ff5d89db3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:08.124419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9df59-9111-473e-8b94-99ff5d89db3d None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:08.124923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9df59-9111-473e-8b94-99ff5d89db3d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:08.125399 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9df59-9111-473e-8b94-99ff5d89db3d None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:08.126318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9df59-9111-473e-8b94-99ff5d89db3d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:08.126906 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9df59-9111-473e-8b94-99ff5d89db3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:08.128518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c9df59-9111-473e-8b94-99ff5d89db3d None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:08.134312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 480/942] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 3 12:41:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:08.145237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1187d40-b62b-4488-ac0d-b4cdaf909881 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:08.208786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:08.243953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1187d40-b62b-4488-ac0d-b4cdaf909881 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:08.279050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261e61f-7ba6-4438-9657-ecb2a823f7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:08.279673 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261e61f-7ba6-4438-9657-ecb2a823f7ec None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:08.280465 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261e61f-7ba6-4438-9657-ecb2a823f7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:08.280746 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261e61f-7ba6-4438-9657-ecb2a823f7ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:08.281501 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261e61f-7ba6-4438-9657-ecb2a823f7ec None admin] RBAC: Adding query filter params (interface=public, service_id=6bf74688115f423f9921ab5a52215816, region_id=RegionOne) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:08.281812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261e61f-7ba6-4438-9657-ecb2a823f7ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:08.283205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261e61f-7ba6-4438-9657-ecb2a823f7ec None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:08.288725 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 463/943] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 3 12:41:08 2022] GET /identity/v3/endpoints?service_id=6bf74688115f423f9921ab5a52215816&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:08.294184 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9df59-9111-473e-8b94-99ff5d89db3d None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:08.331562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:08.400727 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c9df59-9111-473e-8b94-99ff5d89db3d None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:08.439862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb959714-925c-499a-91bf-1e18729b4dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:08.441279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb959714-925c-499a-91bf-1e18729b4dce None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:08.441769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb959714-925c-499a-91bf-1e18729b4dce None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:08.442248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb959714-925c-499a-91bf-1e18729b4dce None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:08.443176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb959714-925c-499a-91bf-1e18729b4dce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:08.444741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb959714-925c-499a-91bf-1e18729b4dce None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:08.451493 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 481/944] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 3 12:41:08 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:41:08.653472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87719000-07a9-49b8-821b-963396c07494 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:08.653677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87719000-07a9-49b8-821b-963396c07494 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:08.653954 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87719000-07a9-49b8-821b-963396c07494 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:08.656020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 464/945] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:08.661210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f44fb-9fd8-4cab-a05c-b9ad3083a401 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:08.661654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f44fb-9fd8-4cab-a05c-b9ad3083a401 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:08.662090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03f44fb-9fd8-4cab-a05c-b9ad3083a401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:08.680955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-d03f44fb-9fd8-4cab-a05c-b9ad3083a401 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:08.682359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d03f44fb-9fd8-4cab-a05c-b9ad3083a401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:08.750874 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 482/946] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:08.774261 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87719000-07a9-49b8-821b-963396c07494 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:08.896676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87719000-07a9-49b8-821b-963396c07494 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:08.927717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:08.933699 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0f1fc-4917-4c3c-8163-183d10de009c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:08.934665 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0f1fc-4917-4c3c-8163-183d10de009c None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:08.934991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0f1fc-4917-4c3c-8163-183d10de009c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:08.935257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c0f1fc-4917-4c3c-8163-183d10de009c None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:08.935997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0f1fc-4917-4c3c-8163-183d10de009c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:08.937454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c0f1fc-4917-4c3c-8163-183d10de009c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:08.940320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: WARNING keystone.server.flask.common [None req-10c0f1fc-4917-4c3c-8163-183d10de009c None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 03 12:41:08.964344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.sql.core [None req-10c0f1fc-4917-4c3c-8163-183d10de009c None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 03 12:41:08.964344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 03 12:41:08.964344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [parameters: {'id': 'a4cca83e008a42509f9245e3f20c7b15', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 03 12:41:08.964344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80306) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 03 12:41:08.981985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-10c0f1fc-4917-4c3c-8163-183d10de009c None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a4cca83e008a42509f9245e3f20c7b15.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a4cca83e008a42509f9245e3f20c7b15. Aug 03 12:41:08.981985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:08.981985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 12:41:08.981985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 12:41:08.981985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 12:41:08.981985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 12:41:08.981985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 12:41:08.981985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 12:41:08.981985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 12:41:08.981985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application conn.query(q) Aug 03 12:41:08.981985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 12:41:08.981985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 12:41:08.981985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application result.read() Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 03 12:41:08.982840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application next(self.gen) Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application yield resource Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application next(self.gen) Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self.session.rollback() Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self.force_reraise() Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise self.value Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application session.commit() Aug 03 12:41:08.983841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._prepare_impl() Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self.session.flush() Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._flush(objects) Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application compat.raise_( Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application flush_context.execute() Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 03 12:41:08.984735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rec.execute(self) Aug 03 12:41:08.985658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 03 12:41:08.985658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 03 12:41:08.985658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 03 12:41:08.985658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 03 12:41:08.985658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 03 12:41:08.985658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 03 12:41:08.985658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 03 12:41:08.985658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 03 12:41:08.985658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 03 12:41:08.985658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 03 12:41:08.985658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 03 12:41:08.985658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 03 12:41:08.985658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 03 12:41:08.985658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 03 12:41:08.985658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application conn.query(q) Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application result.read() Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 12:41:08.986469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 12:41:08.989389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 12:41:08.989389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 12:41:08.989389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 12:41:08.989389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 12:41:08.989389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 12:41:08.989389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 12:41:08.989389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 03 12:41:08.989389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 03 12:41:08.989389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [parameters: {'id': 'a4cca83e008a42509f9245e3f20c7b15', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 03 12:41:08.989389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 03 12:41:08.989389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:08.989389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:08.989389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:08.989389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:08.989389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:08.996337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:09.002202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:09.002202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 03 12:41:09.002202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 03 12:41:09.002202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:09.002202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:09.002202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 03 12:41:09.002202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 03 12:41:09.002202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id a4cca83e008a42509f9245e3f20c7b15. Aug 03 12:41:09.002202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:09.002202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 465/947] 10.222.0.65 () {60 vars in 1140 bytes} [Wed Aug 3 12:41:08 2022] POST /identity/v3/projects => generated 262 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:09.003510 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f44fb-9fd8-4cab-a05c-b9ad3083a401 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:09.035824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:09.109388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03f44fb-9fd8-4cab-a05c-b9ad3083a401 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:09.150015 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae735646-668c-4316-8bfc-c308a3ee44ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:09.150717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae735646-668c-4316-8bfc-c308a3ee44ea None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:09.150980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae735646-668c-4316-8bfc-c308a3ee44ea None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:09.151256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae735646-668c-4316-8bfc-c308a3ee44ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:09.158251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae735646-668c-4316-8bfc-c308a3ee44ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:09.159732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae735646-668c-4316-8bfc-c308a3ee44ea None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:09.171178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-ae735646-668c-4316-8bfc-c308a3ee44ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:41:09.171178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:09.171178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:09.171178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:09.171178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:09.171178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:09.171178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:09.171178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:09.171178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:09.171178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:09.171178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:09.171178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:09.171178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:41:09.171178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:41:09.171178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:09.172067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:09.173012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:41:09.173012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:41:09.173012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:41:09.173012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:41:09.173012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:41:09.173012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:41:09.173012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:41:09.173012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:09.173012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 483/948] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:41:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:09.180069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0f1fc-4917-4c3c-8163-183d10de009c None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:09.286891 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c0f1fc-4917-4c3c-8163-183d10de009c None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:09.323856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41ce52-ec22-4235-bb5b-af7996058149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:09.324860 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41ce52-ec22-4235-bb5b-af7996058149 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:09.325351 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41ce52-ec22-4235-bb5b-af7996058149 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:09.325884 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41ce52-ec22-4235-bb5b-af7996058149 None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:09.326913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41ce52-ec22-4235-bb5b-af7996058149 None admin] RBAC: Adding query filter params (name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:09.327446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41ce52-ec22-4235-bb5b-af7996058149 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:09.329283 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41ce52-ec22-4235-bb5b-af7996058149 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:09.353639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 466/949] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 3 12:41:09 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:09.739675 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360828b-b9db-428f-80e7-c32b78ede08a None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:09.740739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360828b-b9db-428f-80e7-c32b78ede08a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:09.741231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360828b-b9db-428f-80e7-c32b78ede08a None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:09.743069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 484/950] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:09.748827 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aeed3-2668-4bb6-8088-426a28fbbb1d None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:09.749365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aeed3-2668-4bb6-8088-426a28fbbb1d None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:09.749849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6aeed3-2668-4bb6-8088-426a28fbbb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:09.768760 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-2c6aeed3-2668-4bb6-8088-426a28fbbb1d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:09.770795 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2c6aeed3-2668-4bb6-8088-426a28fbbb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:09.819100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:09.833447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee39905-f5c1-4d99-bb94-87f0d5080267 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:09.833743 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee39905-f5c1-4d99-bb94-87f0d5080267 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:09.834010 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee39905-f5c1-4d99-bb94-87f0d5080267 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:09.835663 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 485/951] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:41:09.840585 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5295b-cc98-4bd9-a943-c25ea27cd91b None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:09.840858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5295b-cc98-4bd9-a943-c25ea27cd91b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:09.841349 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5295b-cc98-4bd9-a943-c25ea27cd91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:09.850244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 467/952] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:09.872185 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-73f5295b-cc98-4bd9-a943-c25ea27cd91b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:09.873761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-73f5295b-cc98-4bd9-a943-c25ea27cd91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:09.876818 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6aeed3-2668-4bb6-8088-426a28fbbb1d None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:09.978505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 486/953] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:09.996158 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5295b-cc98-4bd9-a943-c25ea27cd91b None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:10.001188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6aeed3-2668-4bb6-8088-426a28fbbb1d None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:10.022788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:10.050427 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bbebd-95d8-4511-bd71-1d6eb4c4d653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:10.051927 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bbebd-95d8-4511-bd71-1d6eb4c4d653 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:10.052414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bbebd-95d8-4511-bd71-1d6eb4c4d653 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:10.052906 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49bbebd-95d8-4511-bd71-1d6eb4c4d653 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:10.054376 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49bbebd-95d8-4511-bd71-1d6eb4c4d653 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:10.056207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49bbebd-95d8-4511-bd71-1d6eb4c4d653 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-d49bbebd-95d8-4511-bd71-1d6eb4c4d653 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:10.069062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:10.070115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:10.070115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:10.070115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:10.070115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:10.070115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:10.070115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:10.070115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:41:10.070115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:41:10.070115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:41:10.070115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:41:10.070115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:10.070115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:10.070115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:10.070115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:10.070115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:10.070115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:10.071159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:10.071159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:10.071159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:10.071159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:10.071159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:10.071159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:10.071159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:10.071159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:10.071159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:10.071159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:10.071159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:41:10.071159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:10.071159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 12:41:10.071159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:10.072616 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 468/954] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:41:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:10.078086 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bbebd-95d8-4511-bd71-1d6eb4c4d653 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:10.125787 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:10.138241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5295b-cc98-4bd9-a943-c25ea27cd91b None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:10.188342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646413f-cf61-4aee-91d5-ce0962da802d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:10.190107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646413f-cf61-4aee-91d5-ce0962da802d None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:10.190634 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646413f-cf61-4aee-91d5-ce0962da802d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:10.191118 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4646413f-cf61-4aee-91d5-ce0962da802d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:10.191772 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49bbebd-95d8-4511-bd71-1d6eb4c4d653 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:10.209195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4646413f-cf61-4aee-91d5-ce0962da802d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:41:10.216285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-4646413f-cf61-4aee-91d5-ce0962da802d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 12:41:10.216285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:10.216285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:10.216285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:10.216285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:10.216285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:10.216285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:10.216285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:10.216285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:10.216285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:10.216285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:10.216285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:10.216285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:41:10.216285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:41:10.216285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:41:10.216285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:41:10.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:41:10.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:41:10.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:41:10.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:41:10.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:10.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:10.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:10.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:10.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:10.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:10.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:10.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:10.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:10.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:10.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:10.217299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:10.218890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:10.218890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:10.218890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:10.218890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:10.218890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:41:10.218890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:41:10.218890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:41:10.218890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:41:10.218890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:41:10.218890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:41:10.218890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 12:41:10.218890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:10.220352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 487/955] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 3 12:41:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:10.226018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646413f-cf61-4aee-91d5-ce0962da802d None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:10.234434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48155a66-cbd0-4e07-b566-770024acfed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:10.235087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48155a66-cbd0-4e07-b566-770024acfed0 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:10.235368 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48155a66-cbd0-4e07-b566-770024acfed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:10.235641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48155a66-cbd0-4e07-b566-770024acfed0 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:10.236327 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48155a66-cbd0-4e07-b566-770024acfed0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:10.236629 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48155a66-cbd0-4e07-b566-770024acfed0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:10.238132 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48155a66-cbd0-4e07-b566-770024acfed0 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:10.250688 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 469/956] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 3 12:41:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:10.256920 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48155a66-cbd0-4e07-b566-770024acfed0 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:10.338901 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:10.353467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4646413f-cf61-4aee-91d5-ce0962da802d None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:10.378011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48155a66-cbd0-4e07-b566-770024acfed0 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:10.387569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:10.392190 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872dcfd-3de8-4856-84d6-5d1e833b184a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:10.392821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872dcfd-3de8-4856-84d6-5d1e833b184a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:10.393103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872dcfd-3de8-4856-84d6-5d1e833b184a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:10.393379 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872dcfd-3de8-4856-84d6-5d1e833b184a None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:10.394223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872dcfd-3de8-4856-84d6-5d1e833b184a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:10.394569 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872dcfd-3de8-4856-84d6-5d1e833b184a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:10.395945 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872dcfd-3de8-4856-84d6-5d1e833b184a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:10.406527 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 488/957] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 3 12:41:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:10.411910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872dcfd-3de8-4856-84d6-5d1e833b184a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:10.429150 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969c697-e25b-4fdd-8069-682e894fd6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:10.430097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969c697-e25b-4fdd-8069-682e894fd6dd None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:10.430755 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969c697-e25b-4fdd-8069-682e894fd6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:10.431270 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969c697-e25b-4fdd-8069-682e894fd6dd None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:10.432297 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969c697-e25b-4fdd-8069-682e894fd6dd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:10.432834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969c697-e25b-4fdd-8069-682e894fd6dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:10.434919 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969c697-e25b-4fdd-8069-682e894fd6dd None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:10.445400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 470/958] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 3 12:41:10 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:10.510255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a53b4-96d9-4cd4-bf9a-feb206568b49 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:10.510442 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a53b4-96d9-4cd4-bf9a-feb206568b49 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:10.510770 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a53b4-96d9-4cd4-bf9a-feb206568b49 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:10.512414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 471/959] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:41:10.518556 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e115f87-d1f5-47b3-a02e-5357098173b3 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:10.518874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e115f87-d1f5-47b3-a02e-5357098173b3 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:10.519124 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e115f87-d1f5-47b3-a02e-5357098173b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:10.529225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872dcfd-3de8-4856-84d6-5d1e833b184a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:10.540380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-7e115f87-d1f5-47b3-a02e-5357098173b3 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:10.542219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-7e115f87-d1f5-47b3-a02e-5357098173b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:10.574961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1bf895-633b-4a0c-84ae-8796c061ef23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:10.576638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bf895-633b-4a0c-84ae-8796c061ef23 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:10.577720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bf895-633b-4a0c-84ae-8796c061ef23 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:10.578164 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bf895-633b-4a0c-84ae-8796c061ef23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:10.593181 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1bf895-633b-4a0c-84ae-8796c061ef23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:10.595042 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1bf895-633b-4a0c-84ae-8796c061ef23 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:10.599241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:10.614985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-ba1bf895-633b-4a0c-84ae-8796c061ef23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:10.614985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:10.614985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:10.614985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:10.614985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:10.614985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:10.614985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:10.614985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:10.614985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:10.614985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:10.614985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:10.614985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:10.614985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:41:10.614985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:41:10.614985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:10.616263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:10.617482 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:41:10.617482 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:41:10.617482 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:10.617482 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:10.620034 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 489/960] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:10.626665 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1bf895-633b-4a0c-84ae-8796c061ef23 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:10.633676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 472/961] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 12:41:10.656339 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e115f87-d1f5-47b3-a02e-5357098173b3 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:10.745049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1bf895-633b-4a0c-84ae-8796c061ef23 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:10.770085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:10.773564 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e115f87-d1f5-47b3-a02e-5357098173b3 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:10.783886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993f720-a4c8-4b16-8340-17ae91ea2d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:10.784803 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993f720-a4c8-4b16-8340-17ae91ea2d50 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:10.785241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993f720-a4c8-4b16-8340-17ae91ea2d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:10.785671 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993f720-a4c8-4b16-8340-17ae91ea2d50 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:10.786680 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4993f720-a4c8-4b16-8340-17ae91ea2d50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:10.787177 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4993f720-a4c8-4b16-8340-17ae91ea2d50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:10.788735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4993f720-a4c8-4b16-8340-17ae91ea2d50 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:10.806686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 490/962] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:41:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:10.807752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993f720-a4c8-4b16-8340-17ae91ea2d50 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:10.811999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b34c3-e22f-4881-ac44-0409a6d6ffe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:10.813080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b34c3-e22f-4881-ac44-0409a6d6ffe2 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:10.813080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b34c3-e22f-4881-ac44-0409a6d6ffe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:10.813498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499b34c3-e22f-4881-ac44-0409a6d6ffe2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:10.814257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b34c3-e22f-4881-ac44-0409a6d6ffe2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:10.816641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499b34c3-e22f-4881-ac44-0409a6d6ffe2 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-499b34c3-e22f-4881-ac44-0409a6d6ffe2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:10.827784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:10.829066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:10.829066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:10.829066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:10.829066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:10.829066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:10.829066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:10.829066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:41:10.829066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:41:10.829066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:41:10.829066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:41:10.829066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:10.829066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:10.829066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:10.829066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:10.829066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:10.829066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:10.830127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:10.830127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:10.830127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:10.830127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:10.830127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:10.830127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:10.830127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:10.830127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:10.830127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:10.830127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:10.830127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:41:10.830127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:10.830127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 12:41:10.830127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:10.831680 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 473/963] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 3 12:41:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:10.837233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b34c3-e22f-4881-ac44-0409a6d6ffe2 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:10.880722 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:10.928997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993f720-a4c8-4b16-8340-17ae91ea2d50 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:10.954093 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499b34c3-e22f-4881-ac44-0409a6d6ffe2 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:10.966269 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0871a6-3b9c-4e79-b921-bef82cd8c31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:10.967194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0871a6-3b9c-4e79-b921-bef82cd8c31d None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:10.967667 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0871a6-3b9c-4e79-b921-bef82cd8c31d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:10.968092 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0871a6-3b9c-4e79-b921-bef82cd8c31d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:10.977644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0871a6-3b9c-4e79-b921-bef82cd8c31d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:10.979544 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0871a6-3b9c-4e79-b921-bef82cd8c31d None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:10.991144 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6827abb-7c14-4388-bc99-44a44848b282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:10.991892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6827abb-7c14-4388-bc99-44a44848b282 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:10.992242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6827abb-7c14-4388-bc99-44a44848b282 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:10.992587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6827abb-7c14-4388-bc99-44a44848b282 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:10.993380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-7d0871a6-3b9c-4e79-b921-bef82cd8c31d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 12:41:10.993380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:10.993380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:10.993380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:10.993380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:10.993380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:10.993380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:10.993380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:10.993380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:10.993380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:10.993380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:10.993380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:10.993380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:41:10.993380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:41:10.993380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:41:10.993380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:10.994516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:10.995707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:10.995707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:10.995707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:41:10.995707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:41:10.995707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:41:10.995707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:41:10.995707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:41:10.995707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:41:10.995707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 03 12:41:10.995707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:10.996517 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6827abb-7c14-4388-bc99-44a44848b282 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:10.996517 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6827abb-7c14-4388-bc99-44a44848b282 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:10.997433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6827abb-7c14-4388-bc99-44a44848b282 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:11.003409 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 491/964] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 3 12:41:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:11.004419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0871a6-3b9c-4e79-b921-bef82cd8c31d None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:11.006948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 474/965] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 3 12:41:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:11.009339 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6827abb-7c14-4388-bc99-44a44848b282 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:11.039437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:11.115238 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0871a6-3b9c-4e79-b921-bef82cd8c31d None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:11.123180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6827abb-7c14-4388-bc99-44a44848b282 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:11.130569 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:11.152423 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75d47a-4acb-441e-a84f-ced4fcde4ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:11.153003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75d47a-4acb-441e-a84f-ced4fcde4ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:11.153283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75d47a-4acb-441e-a84f-ced4fcde4ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:11.153550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b75d47a-4acb-441e-a84f-ced4fcde4ee7 None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:11.154448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b75d47a-4acb-441e-a84f-ced4fcde4ee7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:11.154792 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b75d47a-4acb-441e-a84f-ced4fcde4ee7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:11.156246 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b75d47a-4acb-441e-a84f-ced4fcde4ee7 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:11.162550 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecc2f9-932d-4cde-b87f-c725c3aaec2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:11.163395 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecc2f9-932d-4cde-b87f-c725c3aaec2c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:11.163846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecc2f9-932d-4cde-b87f-c725c3aaec2c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:11.164337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecc2f9-932d-4cde-b87f-c725c3aaec2c None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:11.165287 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecc2f9-932d-4cde-b87f-c725c3aaec2c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:11.165834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecc2f9-932d-4cde-b87f-c725c3aaec2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:11.167370 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecc2f9-932d-4cde-b87f-c725c3aaec2c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:11.173583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 475/966] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 3 12:41:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:11.182515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 492/967] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 3 12:41:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:11.182686 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecc2f9-932d-4cde-b87f-c725c3aaec2c None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:11.182802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75d47a-4acb-441e-a84f-ced4fcde4ee7 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:11.285098 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:11.296084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b75d47a-4acb-441e-a84f-ced4fcde4ee7 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:11.296250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecc2f9-932d-4cde-b87f-c725c3aaec2c None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:11.333973 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcb03c-fcf5-4c8f-97e2-7cadd5753c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:11.334963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcb03c-fcf5-4c8f-97e2-7cadd5753c4a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:11.335403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcb03c-fcf5-4c8f-97e2-7cadd5753c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:11.335827 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcb03c-fcf5-4c8f-97e2-7cadd5753c4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:11.339340 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ccba28-038f-4e74-8a0d-40c5a34161b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:11.340840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccba28-038f-4e74-8a0d-40c5a34161b6 None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:11.341127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccba28-038f-4e74-8a0d-40c5a34161b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:11.341398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccba28-038f-4e74-8a0d-40c5a34161b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:11.342161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ccba28-038f-4e74-8a0d-40c5a34161b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:11.344069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ccba28-038f-4e74-8a0d-40c5a34161b6 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:11.375814 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 476/968] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:41:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:41:11.390116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:11.424253 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bcb03c-fcf5-4c8f-97e2-7cadd5753c4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:11.425817 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bcb03c-fcf5-4c8f-97e2-7cadd5753c4a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:11.476909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276afd4-c430-487c-af93-c5a52bb64bf6 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:11.477260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276afd4-c430-487c-af93-c5a52bb64bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:11.477476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276afd4-c430-487c-af93-c5a52bb64bf6 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:11.479173 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 477/969] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:41:11.485087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fd26e-82b5-41c1-a440-2079536573d3 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:11.485395 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fd26e-82b5-41c1-a440-2079536573d3 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:11.485647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fd26e-82b5-41c1-a440-2079536573d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:11.504735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-1c2fd26e-82b5-41c1-a440-2079536573d3 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:11.506040 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1c2fd26e-82b5-41c1-a440-2079536573d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:11.548481 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 493/970] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:41:11 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:41:11.553910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcb03c-fcf5-4c8f-97e2-7cadd5753c4a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:11.604218 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 478/971] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 12:41:11.627850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fd26e-82b5-41c1-a440-2079536573d3 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:11.638591 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:11.672376 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcb03c-fcf5-4c8f-97e2-7cadd5753c4a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:11.707897 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:11.713607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5995678-7b0f-4917-93e8-8a93a444df70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:11.714465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5995678-7b0f-4917-93e8-8a93a444df70 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:11.714989 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5995678-7b0f-4917-93e8-8a93a444df70 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:11.715416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5995678-7b0f-4917-93e8-8a93a444df70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:11.729574 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5995678-7b0f-4917-93e8-8a93a444df70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:11.731443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5995678-7b0f-4917-93e8-8a93a444df70 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:11.734300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fd26e-82b5-41c1-a440-2079536573d3 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:11.748506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-d5995678-7b0f-4917-93e8-8a93a444df70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:11.748506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:11.748506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:11.748506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:11.748506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:11.748506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:11.748506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:11.748506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:11.748506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:11.748506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:11.748506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:11.748506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:11.748506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:41:11.748506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:41:11.748506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:11.749824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:11.750875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:41:11.750875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:41:11.750875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:11.750875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:11.753218 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 494/972] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:11.758299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5995678-7b0f-4917-93e8-8a93a444df70 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:11.772232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e53cc-0e5a-4289-87a4-05c416738c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:11.772970 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e53cc-0e5a-4289-87a4-05c416738c26 None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:11.773464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e53cc-0e5a-4289-87a4-05c416738c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:11.773708 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e53cc-0e5a-4289-87a4-05c416738c26 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:11.774506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e53cc-0e5a-4289-87a4-05c416738c26 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:11.776083 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89e53cc-0e5a-4289-87a4-05c416738c26 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:11.792905 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.sql.core [None req-d89e53cc-0e5a-4289-87a4-05c416738c26 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 03 12:41:11.792905 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 03 12:41:11.792905 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [parameters: {'user_id': '2dc21082e6504ea5974f8ff6608acbb5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 03 12:41:11.792905 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80306) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 03 12:41:11.801060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-d89e53cc-0e5a-4289-87a4-05c416738c26 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 03 12:41:11.801060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:11.801060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 12:41:11.801060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 12:41:11.801060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 12:41:11.801060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 12:41:11.801060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 12:41:11.801060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 12:41:11.801060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 12:41:11.801060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application conn.query(q) Aug 03 12:41:11.801060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 12:41:11.801060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 12:41:11.801060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 12:41:11.801060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application result.read() Aug 03 12:41:11.801060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 12:41:11.802018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application next(self.gen) Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application yield resource Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application next(self.gen) Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self.session.rollback() Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self.force_reraise() Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise self.value Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application session.commit() Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 03 12:41:11.803060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._prepare_impl() Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self.session.flush() Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._flush(objects) Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application compat.raise_( Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application flush_context.execute() Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rec.execute(self) Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 03 12:41:11.804013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 03 12:41:11.804956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 03 12:41:11.804956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 03 12:41:11.804956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 03 12:41:11.804956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 03 12:41:11.804956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 03 12:41:11.804956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 03 12:41:11.804956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 03 12:41:11.804956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 03 12:41:11.804956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 03 12:41:11.804956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 03 12:41:11.804956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 03 12:41:11.804956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 03 12:41:11.804956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 03 12:41:11.804956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 03 12:41:11.804956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 03 12:41:11.804956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception Aug 03 12:41:11.805855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 03 12:41:11.805855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 03 12:41:11.805855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 03 12:41:11.805855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 03 12:41:11.805855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 03 12:41:11.805855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application result = self._query(query) Aug 03 12:41:11.805855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 03 12:41:11.805855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application conn.query(q) Aug 03 12:41:11.805855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 03 12:41:11.805855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 03 12:41:11.805855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 03 12:41:11.805855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application result.read() Aug 03 12:41:11.805855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 03 12:41:11.805855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 03 12:41:11.805855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 03 12:41:11.805855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 03 12:41:11.813786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 03 12:41:11.813786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 03 12:41:11.813786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 03 12:41:11.813786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 03 12:41:11.813786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 03 12:41:11.813786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 03 12:41:11.813786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [parameters: {'user_id': '2dc21082e6504ea5974f8ff6608acbb5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 03 12:41:11.813786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 03 12:41:11.813786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:11.813786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:11.813786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:11.813786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:11.813786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:11.813786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:11.813786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 03 12:41:11.816000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 03 12:41:11.818219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 03 12:41:11.818219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 03 12:41:11.818219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 03 12:41:11.818219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 03 12:41:11.818219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 03 12:41:11.818219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:11.818219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 479/973] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 3 12:41:11 2022] POST /identity/v3/users => generated 160 bytes in 174 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:11.818219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e53cc-0e5a-4289-87a4-05c416738c26 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:11.892390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5995678-7b0f-4917-93e8-8a93a444df70 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:11.919130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:11.933279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf2448-541a-43fa-8af1-b5c5f4b8bf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:11.933875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89e53cc-0e5a-4289-87a4-05c416738c26 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:11.934819 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf2448-541a-43fa-8af1-b5c5f4b8bf35 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:11.935326 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf2448-541a-43fa-8af1-b5c5f4b8bf35 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:11.935885 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf2448-541a-43fa-8af1-b5c5f4b8bf35 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:11.936943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf2448-541a-43fa-8af1-b5c5f4b8bf35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:11.937456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf2448-541a-43fa-8af1-b5c5f4b8bf35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:11.939070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf2448-541a-43fa-8af1-b5c5f4b8bf35 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:11.953850 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 495/974] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:41:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:11.961333 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf2448-541a-43fa-8af1-b5c5f4b8bf35 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:11.972925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd38b83-eb09-4fec-b7cf-897a8cd9bcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:11.973569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd38b83-eb09-4fec-b7cf-897a8cd9bcd5 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:11.973851 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd38b83-eb09-4fec-b7cf-897a8cd9bcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:11.974120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd38b83-eb09-4fec-b7cf-897a8cd9bcd5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:11.977137 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:11.982453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd38b83-eb09-4fec-b7cf-897a8cd9bcd5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:11.984602 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd38b83-eb09-4fec-b7cf-897a8cd9bcd5 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:11.998310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-8fd38b83-eb09-4fec-b7cf-897a8cd9bcd5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 12:41:11.998310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:11.998310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:11.998310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:11.998310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:11.998310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:11.998310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:11.998310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:11.998310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:11.998310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:11.998310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:11.998310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:11.998310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:41:11.998310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:41:11.998310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:41:11.998310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:12.000107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:12.001276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:12.001276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:12.001276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:41:12.001276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:41:12.001276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:41:12.001276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:41:12.001276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:41:12.001276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:41:12.001276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 03 12:41:12.001276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:12.003778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 480/975] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:41:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:12.011442 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd38b83-eb09-4fec-b7cf-897a8cd9bcd5 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:12.082067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf2448-541a-43fa-8af1-b5c5f4b8bf35 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:12.118401 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7254f0d-f59b-40cc-991b-74473cb1f303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:12.119364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7254f0d-f59b-40cc-991b-74473cb1f303 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:12.119854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7254f0d-f59b-40cc-991b-74473cb1f303 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:12.120313 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7254f0d-f59b-40cc-991b-74473cb1f303 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:12.128034 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7254f0d-f59b-40cc-991b-74473cb1f303 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:12.129735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7254f0d-f59b-40cc-991b-74473cb1f303 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:12.142465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-a7254f0d-f59b-40cc-991b-74473cb1f303 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:41:12.142465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:12.142465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:12.142465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:12.142465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:12.142465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:12.142465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:12.142465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:12.142465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:12.142465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:12.142465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:12.142465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:12.142465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:41:12.142465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:41:12.142465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:12.143470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:12.144466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:41:12.144466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:41:12.144466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:41:12.144466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:41:12.144466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:41:12.144466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:41:12.144466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:41:12.144466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:12.151444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 496/976] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:41:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:12.152510 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7254f0d-f59b-40cc-991b-74473cb1f303 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:12.155201 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd38b83-eb09-4fec-b7cf-897a8cd9bcd5 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:12.196747 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aaace-a7d6-491c-bb65-4c790bfd4e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:12.197372 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aaace-a7d6-491c-bb65-4c790bfd4e40 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:12.197656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aaace-a7d6-491c-bb65-4c790bfd4e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:12.197934 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0aaace-a7d6-491c-bb65-4c790bfd4e40 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:12.198847 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0aaace-a7d6-491c-bb65-4c790bfd4e40 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:12.199156 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0aaace-a7d6-491c-bb65-4c790bfd4e40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:12.200687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0aaace-a7d6-491c-bb65-4c790bfd4e40 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:12.204055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:12.249578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 481/977] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 3 12:41:12 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:12.261973 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7254f0d-f59b-40cc-991b-74473cb1f303 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:12.301444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c513d61c-e503-4520-a5e3-2062ce8ee36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:12.302458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513d61c-e503-4520-a5e3-2062ce8ee36f None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:12.302999 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513d61c-e503-4520-a5e3-2062ce8ee36f None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:12.303509 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513d61c-e503-4520-a5e3-2062ce8ee36f None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:12.304518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513d61c-e503-4520-a5e3-2062ce8ee36f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:12.305015 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513d61c-e503-4520-a5e3-2062ce8ee36f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:12.306677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c513d61c-e503-4520-a5e3-2062ce8ee36f None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:12.320952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 497/978] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:41:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:12.332580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aaace-a7d6-491c-bb65-4c790bfd4e40 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:12.396280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:12.449749 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0aaace-a7d6-491c-bb65-4c790bfd4e40 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:12.485820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49223c4-6591-4fa7-9fa0-247475f421ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:12.486450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49223c4-6591-4fa7-9fa0-247475f421ae None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:12.486762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49223c4-6591-4fa7-9fa0-247475f421ae None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:12.487028 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49223c4-6591-4fa7-9fa0-247475f421ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:12.487820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49223c4-6591-4fa7-9fa0-247475f421ae None admin] RBAC: Adding query filter params (role.id=b9b9695215b144d4b974ba1266bb18c1, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=3ae6eeb3d4f94925a5e1dd497d352d2e) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:12.488127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49223c4-6591-4fa7-9fa0-247475f421ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:12.489622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49223c4-6591-4fa7-9fa0-247475f421ae None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:12.496282 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 482/979] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:41:12 2022] GET /identity/v3/role_assignments?user.id=3ae6eeb3d4f94925a5e1dd497d352d2e&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=b9b9695215b144d4b974ba1266bb18c1 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:12.603052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475ab9c-53cf-40a9-b0c1-24608cd1dfed None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:12.603231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475ab9c-53cf-40a9-b0c1-24608cd1dfed None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:12.603513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475ab9c-53cf-40a9-b0c1-24608cd1dfed None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:12.605031 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 498/980] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:12.611087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b66c9-c684-47dd-9bea-4035ae67aebc None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:12.611387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b66c9-c684-47dd-9bea-4035ae67aebc None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:12.611653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b66c9-c684-47dd-9bea-4035ae67aebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:12.631216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-c03b66c9-c684-47dd-9bea-4035ae67aebc None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:12.632800 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c03b66c9-c684-47dd-9bea-4035ae67aebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:12.707090 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 483/981] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:12.728758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475ab9c-53cf-40a9-b0c1-24608cd1dfed None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:12.830844 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475ab9c-53cf-40a9-b0c1-24608cd1dfed None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:12.833154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:12.871363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948a41a-3f19-448a-9624-79928c32c4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:12.871905 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948a41a-3f19-448a-9624-79928c32c4d7 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:12.872186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948a41a-3f19-448a-9624-79928c32c4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:12.872458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948a41a-3f19-448a-9624-79928c32c4d7 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:12.874528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948a41a-3f19-448a-9624-79928c32c4d7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:12.874655 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948a41a-3f19-448a-9624-79928c32c4d7 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:12.894530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 499/982] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 3 12:41:12 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:41:13.460378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d44cfc-6ea2-4821-9756-fddf1bbcd345 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:13.460900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d44cfc-6ea2-4821-9756-fddf1bbcd345 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:13.460900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d44cfc-6ea2-4821-9756-fddf1bbcd345 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:13.462429 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 484/983] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:13.468065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6562d1-18b2-41d1-aaed-db96b89df6b6 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:13.468356 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6562d1-18b2-41d1-aaed-db96b89df6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:13.468631 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6562d1-18b2-41d1-aaed-db96b89df6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:13.486699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-6a6562d1-18b2-41d1-aaed-db96b89df6b6 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:13.487877 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-6a6562d1-18b2-41d1-aaed-db96b89df6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:13.562178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 500/984] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:13 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:13.584513 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d44cfc-6ea2-4821-9756-fddf1bbcd345 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:13.638385 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:13.691750 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d44cfc-6ea2-4821-9756-fddf1bbcd345 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:13.725266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ba425-f375-4769-8b87-82645d538214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:13.725833 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ba425-f375-4769-8b87-82645d538214 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:13.726173 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ba425-f375-4769-8b87-82645d538214 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:13.726452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ba425-f375-4769-8b87-82645d538214 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:13.727112 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ba425-f375-4769-8b87-82645d538214 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:13.728465 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ba425-f375-4769-8b87-82645d538214 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-533ba425-f375-4769-8b87-82645d538214 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:13.737243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:13.738258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:13.738258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:13.738258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:13.738258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:13.738258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:13.738258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:13.738258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:41:13.738258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:41:13.738258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:41:13.738258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:41:13.738258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:13.738258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:13.738258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:13.738258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:13.738258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:13.738258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:13.744100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:13.744100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:13.744100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:13.744100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:13.744100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:13.744100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:13.744100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:13.744100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:13.744100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:13.744100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:13.744100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:41:13.744100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:13.744100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 12:41:13.744100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:13.744100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 485/985] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 3 12:41:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:13.746524 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6562d1-18b2-41d1-aaed-db96b89df6b6 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:13.834779 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:13.846522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6562d1-18b2-41d1-aaed-db96b89df6b6 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:13.888655 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93488082-b376-490c-9074-5938609d79b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:13.889324 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93488082-b376-490c-9074-5938609d79b4 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:13.889599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93488082-b376-490c-9074-5938609d79b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:13.889869 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93488082-b376-490c-9074-5938609d79b4 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:13.890572 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93488082-b376-490c-9074-5938609d79b4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:13.890879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93488082-b376-490c-9074-5938609d79b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:13.892231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93488082-b376-490c-9074-5938609d79b4 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:13.901568 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 501/986] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 3 12:41:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:13.903079 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ba425-f375-4769-8b87-82645d538214 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:14.009442 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ba425-f375-4769-8b87-82645d538214 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:14.022572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:14.044585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c46c68-6207-4cad-873a-beba78fa3127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:14.045275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c46c68-6207-4cad-873a-beba78fa3127 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:14.046048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c46c68-6207-4cad-873a-beba78fa3127 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:14.046321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c46c68-6207-4cad-873a-beba78fa3127 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:14.047036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c46c68-6207-4cad-873a-beba78fa3127 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:14.047343 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c46c68-6207-4cad-873a-beba78fa3127 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:14.048734 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c46c68-6207-4cad-873a-beba78fa3127 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:14.058426 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 486/987] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 3 12:41:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:14.060249 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93488082-b376-490c-9074-5938609d79b4 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:14.159924 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93488082-b376-490c-9074-5938609d79b4 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:14.193920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8ce09a-2557-49bc-a91d-e266d64c17f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:14.194567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ce09a-2557-49bc-a91d-e266d64c17f1 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:14.194851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ce09a-2557-49bc-a91d-e266d64c17f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:14.195124 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8ce09a-2557-49bc-a91d-e266d64c17f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:14.195879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ce09a-2557-49bc-a91d-e266d64c17f1 None admin] RBAC: Adding query filter params (interface=admin, service_id=6bf74688115f423f9921ab5a52215816, region_id=RegionOne) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:14.196189 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ce09a-2557-49bc-a91d-e266d64c17f1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:14.197531 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8ce09a-2557-49bc-a91d-e266d64c17f1 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:14.207015 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 502/988] 10.222.0.65 () {56 vars in 1249 bytes} [Wed Aug 3 12:41:14 2022] GET /identity/v3/endpoints?service_id=6bf74688115f423f9921ab5a52215816&interface=admin®ion_id=RegionOne => generated 193 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:14.208586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c46c68-6207-4cad-873a-beba78fa3127 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:14.310855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c46c68-6207-4cad-873a-beba78fa3127 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:14.344453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4089a03-1b31-43ba-8c1f-550ca3c4ba46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:14.345073 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4089a03-1b31-43ba-8c1f-550ca3c4ba46 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:14.345427 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4089a03-1b31-43ba-8c1f-550ca3c4ba46 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:14.345712 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4089a03-1b31-43ba-8c1f-550ca3c4ba46 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:14.346363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4089a03-1b31-43ba-8c1f-550ca3c4ba46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:14.347780 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4089a03-1b31-43ba-8c1f-550ca3c4ba46 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:14.354124 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 487/989] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 3 12:41:14 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:41:14.538176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50387b02-3de4-48c1-960c-48d96dc9f37b None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:14.538665 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50387b02-3de4-48c1-960c-48d96dc9f37b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:14.538772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50387b02-3de4-48c1-960c-48d96dc9f37b None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:14.540268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 503/990] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:14.545186 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b46266-17b8-4b32-b8ec-dc2cf461e799 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:14.546255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b46266-17b8-4b32-b8ec-dc2cf461e799 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:14.546340 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b46266-17b8-4b32-b8ec-dc2cf461e799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:14.565558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-d5b46266-17b8-4b32-b8ec-dc2cf461e799 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:14.566712 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d5b46266-17b8-4b32-b8ec-dc2cf461e799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:14.602255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:14.635578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 488/991] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:14.658406 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50387b02-3de4-48c1-960c-48d96dc9f37b None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:14.678667 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:14.763521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50387b02-3de4-48c1-960c-48d96dc9f37b None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:14.798251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d68a7b-6d4e-4f90-80ec-95df0004fcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:14.798928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d68a7b-6d4e-4f90-80ec-95df0004fcbf None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:14.799211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d68a7b-6d4e-4f90-80ec-95df0004fcbf None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:14.799485 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d68a7b-6d4e-4f90-80ec-95df0004fcbf None admin] PATH_INFO: `/v3/services/image` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:14.800109 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d68a7b-6d4e-4f90-80ec-95df0004fcbf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:14.801471 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d68a7b-6d4e-4f90-80ec-95df0004fcbf None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-b7d68a7b-6d4e-4f90-80ec-95df0004fcbf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:14.810895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:14.811990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:14.811990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:14.811990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:14.811990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:14.811990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:14.811990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:14.811990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:41:14.811990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:41:14.811990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:41:14.811990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:41:14.811990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:14.811990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:14.811990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:14.811990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:14.811990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:14.811990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:14.813465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:14.813465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:14.813465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:14.813465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:14.813465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:14.813465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:14.813465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:14.813465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:14.813465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:14.813465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:14.813465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:41:14.813465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:14.813465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 12:41:14.813465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:14.813465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 504/992] 10.222.0.65 () {56 vars in 1105 bytes} [Wed Aug 3 12:41:14 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:14.818117 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b46266-17b8-4b32-b8ec-dc2cf461e799 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:14.897942 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a735d88-646b-4927-a02b-34bcbd3b9483 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:14.898466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a735d88-646b-4927-a02b-34bcbd3b9483 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:14.898466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a735d88-646b-4927-a02b-34bcbd3b9483 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:14.899909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 505/993] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:14.904599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903d783-9ae7-4053-97ba-9fa98b617797 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:14.905431 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903d783-9ae7-4053-97ba-9fa98b617797 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:14.905808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903d783-9ae7-4053-97ba-9fa98b617797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:14.923407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-3903d783-9ae7-4053-97ba-9fa98b617797 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:14.924582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-3903d783-9ae7-4053-97ba-9fa98b617797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:14.927816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b46266-17b8-4b32-b8ec-dc2cf461e799 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:14.963044 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998ea11-9b19-4ab1-b578-5dd52e23fdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:14.963663 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998ea11-9b19-4ab1-b578-5dd52e23fdfe None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:14.963942 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998ea11-9b19-4ab1-b578-5dd52e23fdfe None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:14.964212 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998ea11-9b19-4ab1-b578-5dd52e23fdfe None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:14.964894 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998ea11-9b19-4ab1-b578-5dd52e23fdfe None admin] RBAC: Adding query filter params (name=image) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:14.965202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998ea11-9b19-4ab1-b578-5dd52e23fdfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:14.966593 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998ea11-9b19-4ab1-b578-5dd52e23fdfe None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:14.976808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 489/994] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:14.978035 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998ea11-9b19-4ab1-b578-5dd52e23fdfe None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:15.003223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 506/995] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 12:41:15.015274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:15.025396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903d783-9ae7-4053-97ba-9fa98b617797 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:15.039934 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:15.088311 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998ea11-9b19-4ab1-b578-5dd52e23fdfe None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:15.126338 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2e6f2-20c2-497a-ba60-ed5a13b7ed01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:15.126990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2e6f2-20c2-497a-ba60-ed5a13b7ed01 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:15.127270 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2e6f2-20c2-497a-ba60-ed5a13b7ed01 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:15.127545 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2e6f2-20c2-497a-ba60-ed5a13b7ed01 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:15.128223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2e6f2-20c2-497a-ba60-ed5a13b7ed01 None admin] RBAC: Adding query filter params (type=image) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:15.128528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2e6f2-20c2-497a-ba60-ed5a13b7ed01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:15.129956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb2e6f2-20c2-497a-ba60-ed5a13b7ed01 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:15.130812 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903d783-9ae7-4053-97ba-9fa98b617797 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:15.138819 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 490/996] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:14 2022] GET /identity/v3/services?type=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:15.168281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58149196-cdb9-4b4e-b8d3-9b972180b95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:15.168954 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58149196-cdb9-4b4e-b8d3-9b972180b95b None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:15.169234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58149196-cdb9-4b4e-b8d3-9b972180b95b None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:15.169508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58149196-cdb9-4b4e-b8d3-9b972180b95b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:15.170151 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58149196-cdb9-4b4e-b8d3-9b972180b95b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:15.171626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58149196-cdb9-4b4e-b8d3-9b972180b95b None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-58149196-cdb9-4b4e-b8d3-9b972180b95b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:15.181308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:15.182414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:15.182414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:15.182414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:15.182414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:15.182414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:15.182414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:15.182414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:41:15.182414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:41:15.182414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:41:15.182414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:41:15.182414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:15.182414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:15.182414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:15.182414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:15.182414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:15.182414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:15.183331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:15.183331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:15.183331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:15.183331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:15.183331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:15.183331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:15.183331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:15.183331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:15.183331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:15.183331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:15.183331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:41:15.183331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:15.183331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 12:41:15.183331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:15.185222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 507/997] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:41:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:15.189660 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2e6f2-20c2-497a-ba60-ed5a13b7ed01 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:15.297366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:15.302148 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2e6f2-20c2-497a-ba60-ed5a13b7ed01 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:15.338960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af45305-0a71-4d67-91a1-efc03ab67c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:15.339550 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af45305-0a71-4d67-91a1-efc03ab67c74 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:15.339836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af45305-0a71-4d67-91a1-efc03ab67c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:15.340106 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af45305-0a71-4d67-91a1-efc03ab67c74 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:15.340781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af45305-0a71-4d67-91a1-efc03ab67c74 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:15.341082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af45305-0a71-4d67-91a1-efc03ab67c74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:15.342464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af45305-0a71-4d67-91a1-efc03ab67c74 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:15.351904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 491/998] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 3 12:41:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:15.353415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58149196-cdb9-4b4e-b8d3-9b972180b95b None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:15.432686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:15.458406 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58149196-cdb9-4b4e-b8d3-9b972180b95b None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:15.493255 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa6f6a-163c-44ee-8af1-bc530d05ace0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:15.493884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa6f6a-163c-44ee-8af1-bc530d05ace0 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:15.494167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa6f6a-163c-44ee-8af1-bc530d05ace0 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:15.494441 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa6f6a-163c-44ee-8af1-bc530d05ace0 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:15.495150 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa6f6a-163c-44ee-8af1-bc530d05ace0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:15.495459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa6f6a-163c-44ee-8af1-bc530d05ace0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:15.496816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa6f6a-163c-44ee-8af1-bc530d05ace0 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:15.502423 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 508/999] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 3 12:41:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:15.508278 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af45305-0a71-4d67-91a1-efc03ab67c74 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:15.611465 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af45305-0a71-4d67-91a1-efc03ab67c74 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:15.645440 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf3df9-100d-4df4-a8fb-0538d0c323e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:15.646050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf3df9-100d-4df4-a8fb-0538d0c323e2 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:15.646325 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf3df9-100d-4df4-a8fb-0538d0c323e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:15.646623 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf3df9-100d-4df4-a8fb-0538d0c323e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:15.647370 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf3df9-100d-4df4-a8fb-0538d0c323e2 None admin] RBAC: Adding query filter params (interface=public, service_id=9218044a55fe4581960abf213b9bc3d5, region_id=RegionOne) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:15.647674 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf3df9-100d-4df4-a8fb-0538d0c323e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:15.649020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf3df9-100d-4df4-a8fb-0538d0c323e2 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:15.654143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 492/1000] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 3 12:41:15 2022] GET /identity/v3/endpoints?service_id=9218044a55fe4581960abf213b9bc3d5&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:15.659429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa6f6a-163c-44ee-8af1-bc530d05ace0 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:15.757153 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa6f6a-163c-44ee-8af1-bc530d05ace0 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:15.790276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983dee8-4f72-432b-a429-e42397f47b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:15.790921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983dee8-4f72-432b-a429-e42397f47b8e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:15.791199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983dee8-4f72-432b-a429-e42397f47b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:15.791469 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983dee8-4f72-432b-a429-e42397f47b8e None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:15.792108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a983dee8-4f72-432b-a429-e42397f47b8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:15.793453 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a983dee8-4f72-432b-a429-e42397f47b8e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:15.806575 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 509/1001] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 3 12:41:15 2022] GET /identity/v3/services => generated 1727 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:41:16.367883 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a17648-c12d-4f41-b6a4-d68a0a2f4603 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:16.368218 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a17648-c12d-4f41-b6a4-d68a0a2f4603 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:16.368437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a17648-c12d-4f41-b6a4-d68a0a2f4603 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:16.369948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 493/1002] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:16.374908 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b6c34-46a5-41af-9f49-7f7a80456409 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:16.375342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b6c34-46a5-41af-9f49-7f7a80456409 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:16.375435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b6c34-46a5-41af-9f49-7f7a80456409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:16.382653 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:16.395883 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-ae0b6c34-46a5-41af-9f49-7f7a80456409 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:16.397048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ae0b6c34-46a5-41af-9f49-7f7a80456409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:16.462372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 510/1003] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:16.484577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a17648-c12d-4f41-b6a4-d68a0a2f4603 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:16.511899 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:16.589421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a17648-c12d-4f41-b6a4-d68a0a2f4603 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:16.622570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292c3cb-8dc2-4684-b810-77e9d8fde3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:16.623164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292c3cb-8dc2-4684-b810-77e9d8fde3cd None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:16.623915 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292c3cb-8dc2-4684-b810-77e9d8fde3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:16.624194 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292c3cb-8dc2-4684-b810-77e9d8fde3cd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:16.624806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292c3cb-8dc2-4684-b810-77e9d8fde3cd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:16.626152 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292c3cb-8dc2-4684-b810-77e9d8fde3cd None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-0292c3cb-8dc2-4684-b810-77e9d8fde3cd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:16.634923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:16.635901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:16.635901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:16.635901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:16.635901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:16.635901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:16.635901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:16.635901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:41:16.635901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:41:16.635901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:41:16.635901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:41:16.635901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:16.635901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:16.635901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:16.635901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:16.635901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:16.635901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:16.636825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:16.636825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:16.636825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:16.636825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:16.636825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:16.636825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:16.636825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:16.636825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:16.636825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:16.636825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:16.636825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:41:16.636825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:16.636825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 03 12:41:16.636825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:16.638632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 494/1004] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 3 12:41:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:16.642625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b6c34-46a5-41af-9f49-7f7a80456409 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:16.739380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b6c34-46a5-41af-9f49-7f7a80456409 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:16.773538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865be6c-4099-4554-a0c3-0fa68cfdbe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:16.774462 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865be6c-4099-4554-a0c3-0fa68cfdbe04 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:16.774963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865be6c-4099-4554-a0c3-0fa68cfdbe04 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:16.775423 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865be6c-4099-4554-a0c3-0fa68cfdbe04 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:16.776266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865be6c-4099-4554-a0c3-0fa68cfdbe04 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:16.776764 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865be6c-4099-4554-a0c3-0fa68cfdbe04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:16.778405 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8865be6c-4099-4554-a0c3-0fa68cfdbe04 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:16.779646 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:16.787116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 511/1005] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 3 12:41:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:16.791566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292c3cb-8dc2-4684-b810-77e9d8fde3cd None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:16.889311 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:16.899889 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292c3cb-8dc2-4684-b810-77e9d8fde3cd None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:16.937014 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b276fef-a5d7-448b-b79a-fae25e10f487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:16.937658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b276fef-a5d7-448b-b79a-fae25e10f487 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:16.937939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b276fef-a5d7-448b-b79a-fae25e10f487 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:16.938210 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b276fef-a5d7-448b-b79a-fae25e10f487 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:16.938938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b276fef-a5d7-448b-b79a-fae25e10f487 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:16.939232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b276fef-a5d7-448b-b79a-fae25e10f487 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:16.940602 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b276fef-a5d7-448b-b79a-fae25e10f487 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:16.945802 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 495/1006] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 3 12:41:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:16.951281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865be6c-4099-4554-a0c3-0fa68cfdbe04 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:17.050166 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865be6c-4099-4554-a0c3-0fa68cfdbe04 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:17.083658 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e92cd-a53b-4b90-9522-db4184b36771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:17.084276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e92cd-a53b-4b90-9522-db4184b36771 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:17.084562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e92cd-a53b-4b90-9522-db4184b36771 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:17.084838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e92cd-a53b-4b90-9522-db4184b36771 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:17.085558 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e92cd-a53b-4b90-9522-db4184b36771 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:17.086958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e92cd-a53b-4b90-9522-db4184b36771 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:17.110519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 512/1007] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:41:16 2022] POST /identity/v3/endpoints => generated 325 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:41:17.135477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94fb79-bd18-4339-ac68-6bc2f02cb073 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:17.135676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94fb79-bd18-4339-ac68-6bc2f02cb073 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:17.135944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94fb79-bd18-4339-ac68-6bc2f02cb073 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:17.137504 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 496/1008] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:17.143082 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453528a-9f65-4e5c-8b2a-7e171ac9a025 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:17.143398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453528a-9f65-4e5c-8b2a-7e171ac9a025 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:17.143640 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453528a-9f65-4e5c-8b2a-7e171ac9a025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:17.162915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-9453528a-9f65-4e5c-8b2a-7e171ac9a025 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:17.164146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-9453528a-9f65-4e5c-8b2a-7e171ac9a025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:17.226381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:17.249475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 513/1009] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:17 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:17.271357 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94fb79-bd18-4339-ac68-6bc2f02cb073 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:17.377827 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94fb79-bd18-4339-ac68-6bc2f02cb073 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:17.411691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b634d12-5831-4e6f-8f6c-3259ad504294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:17.412219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b634d12-5831-4e6f-8f6c-3259ad504294 None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:17.412499 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b634d12-5831-4e6f-8f6c-3259ad504294 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:17.412771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b634d12-5831-4e6f-8f6c-3259ad504294 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:17.413460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b634d12-5831-4e6f-8f6c-3259ad504294 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:17.414853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b634d12-5831-4e6f-8f6c-3259ad504294 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:17.438509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 497/1010] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 3 12:41:17 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:41:17.812001 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbec893-b058-4d15-8329-2582c128ccd3 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:17.812554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbec893-b058-4d15-8329-2582c128ccd3 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:17.812554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbec893-b058-4d15-8329-2582c128ccd3 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:17.814014 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 514/1011] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:17.818983 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c183a77-566e-4869-a52f-0a4361725b8a None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:17.819382 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c183a77-566e-4869-a52f-0a4361725b8a None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:17.819543 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c183a77-566e-4869-a52f-0a4361725b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:17.838011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-6c183a77-566e-4869-a52f-0a4361725b8a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:17.839231 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6c183a77-566e-4869-a52f-0a4361725b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:17.849303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:17.925264 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 498/1012] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:17 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:17.955458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbec893-b058-4d15-8329-2582c128ccd3 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:18.055654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbec893-b058-4d15-8329-2582c128ccd3 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:18.089933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989d73c-86f5-4c10-8696-d94d1ff19294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:18.090521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989d73c-86f5-4c10-8696-d94d1ff19294 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:18.090806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989d73c-86f5-4c10-8696-d94d1ff19294 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:18.091079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989d73c-86f5-4c10-8696-d94d1ff19294 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:18.091703 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989d73c-86f5-4c10-8696-d94d1ff19294 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:18.093156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989d73c-86f5-4c10-8696-d94d1ff19294 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-2989d73c-86f5-4c10-8696-d94d1ff19294 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:18.102120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:18.103293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:18.103293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:18.103293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:18.103293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:18.103293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:18.103293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:18.103293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:41:18.103293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:41:18.103293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:41:18.103293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:41:18.103293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:18.103293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:18.103293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:18.103293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:18.103293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:18.103293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:18.109012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:18.109012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:18.109012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:18.109012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:18.109012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:18.109012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:18.109012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:18.109012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:18.109012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:18.109012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:18.109012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:41:18.109012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:18.109012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 03 12:41:18.109012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:18.109012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 515/1013] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:41:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:18.111338 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c183a77-566e-4869-a52f-0a4361725b8a None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:18.211878 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c183a77-566e-4869-a52f-0a4361725b8a None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:18.247878 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f589ee-00cd-49fa-90af-5e4dfbda6c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:18.249007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f589ee-00cd-49fa-90af-5e4dfbda6c4a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:18.249286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f589ee-00cd-49fa-90af-5e4dfbda6c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:18.249557 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f589ee-00cd-49fa-90af-5e4dfbda6c4a None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:18.250235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f589ee-00cd-49fa-90af-5e4dfbda6c4a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:18.250565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f589ee-00cd-49fa-90af-5e4dfbda6c4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:18.251925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f589ee-00cd-49fa-90af-5e4dfbda6c4a None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:18.257056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 499/1014] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 3 12:41:18 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:18.262680 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989d73c-86f5-4c10-8696-d94d1ff19294 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:18.313329 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:18.364605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989d73c-86f5-4c10-8696-d94d1ff19294 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:18.397241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fdbc4-be9e-48d9-b086-e7ca95b7548f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:18.397855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fdbc4-be9e-48d9-b086-e7ca95b7548f None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:18.398138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fdbc4-be9e-48d9-b086-e7ca95b7548f None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:18.398411 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fdbc4-be9e-48d9-b086-e7ca95b7548f None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:18.399108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4fdbc4-be9e-48d9-b086-e7ca95b7548f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:18.399415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4fdbc4-be9e-48d9-b086-e7ca95b7548f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:18.400765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4fdbc4-be9e-48d9-b086-e7ca95b7548f None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:18.405662 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 516/1015] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 3 12:41:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:18.411343 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f589ee-00cd-49fa-90af-5e4dfbda6c4a None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:18.415299 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:18.511963 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f589ee-00cd-49fa-90af-5e4dfbda6c4a None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:18.546536 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91e571-ac10-4293-a6de-aff21810a8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:18.547182 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91e571-ac10-4293-a6de-aff21810a8a2 None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:18.547463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91e571-ac10-4293-a6de-aff21810a8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:18.547736 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91e571-ac10-4293-a6de-aff21810a8a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:18.548489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91e571-ac10-4293-a6de-aff21810a8a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:18.549953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc91e571-ac10-4293-a6de-aff21810a8a2 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:18.578515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 500/1016] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:41:18 2022] POST /identity/v3/endpoints => generated 334 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:41:19.128814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0801ef0-2d42-45cd-a1f4-f14cf0d7342e None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:19.129301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0801ef0-2d42-45cd-a1f4-f14cf0d7342e None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:19.129301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0801ef0-2d42-45cd-a1f4-f14cf0d7342e None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:19.130764 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 517/1017] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:19.135732 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac507f4d-3e12-41e9-883f-4f9850424cf0 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:19.136172 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac507f4d-3e12-41e9-883f-4f9850424cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:19.136340 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac507f4d-3e12-41e9-883f-4f9850424cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:19.154437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-ac507f4d-3e12-41e9-883f-4f9850424cf0 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:19.156198 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ac507f4d-3e12-41e9-883f-4f9850424cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:19.212633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:19.238254 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 501/1018] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:19.260208 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0801ef0-2d42-45cd-a1f4-f14cf0d7342e None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:19.363762 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0801ef0-2d42-45cd-a1f4-f14cf0d7342e None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:19.376773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:19.398582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9db81-e15c-43fe-926a-921b13ddeff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:19.399140 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9db81-e15c-43fe-926a-921b13ddeff7 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:19.399417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9db81-e15c-43fe-926a-921b13ddeff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:19.399686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9db81-e15c-43fe-926a-921b13ddeff7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:19.400827 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9db81-e15c-43fe-926a-921b13ddeff7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:19.402238 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9db81-e15c-43fe-926a-921b13ddeff7 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:19.404012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 518/1019] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:19.410187 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac507f4d-3e12-41e9-883f-4f9850424cf0 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:19.452848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b63c2c-2315-4769-98a1-4ddb59b42a07 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:19.453130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b63c2c-2315-4769-98a1-4ddb59b42a07 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:19.453398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b63c2c-2315-4769-98a1-4ddb59b42a07 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:19.454750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 519/1020] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:41:19.459470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cecb20-1c68-4c92-b155-b62ca1bf3f93 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:19.459753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cecb20-1c68-4c92-b155-b62ca1bf3f93 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:19.460026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cecb20-1c68-4c92-b155-b62ca1bf3f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:19.478106 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-27cecb20-1c68-4c92-b155-b62ca1bf3f93 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:19.479317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-27cecb20-1c68-4c92-b155-b62ca1bf3f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:19.513506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac507f4d-3e12-41e9-883f-4f9850424cf0 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:19.547575 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 520/1021] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 12:41:19.548406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1f961-9087-49d2-930a-9dccc1565517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:19.549054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1f961-9087-49d2-930a-9dccc1565517 None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:19.549337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1f961-9087-49d2-930a-9dccc1565517 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:19.549610 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1f961-9087-49d2-930a-9dccc1565517 None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:19.550300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1f961-9087-49d2-930a-9dccc1565517 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:19.551760 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1f961-9087-49d2-930a-9dccc1565517 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:19.570513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cecb20-1c68-4c92-b155-b62ca1bf3f93 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:19.582411 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 502/1022] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 3 12:41:19 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:41:19.680495 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cecb20-1c68-4c92-b155-b62ca1bf3f93 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:19.692992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:19.720687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80de4cd-85d4-4526-affb-35e2bac2b69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:19.721685 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80de4cd-85d4-4526-affb-35e2bac2b69a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:19.722156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80de4cd-85d4-4526-affb-35e2bac2b69a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:19.722670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80de4cd-85d4-4526-affb-35e2bac2b69a None admin] PATH_INFO: `/v3/services/image` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:19.723504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80de4cd-85d4-4526-affb-35e2bac2b69a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:19.725165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80de4cd-85d4-4526-affb-35e2bac2b69a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-d80de4cd-85d4-4526-affb-35e2bac2b69a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:19.735393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:19.736403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:19.736403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:19.736403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:19.736403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:19.736403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:19.736403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:19.736403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:41:19.736403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:41:19.736403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:41:19.736403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:41:19.736403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:19.736403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:19.736403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:19.736403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:19.736403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:19.736403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:19.737254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:19.737254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:19.737254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:19.737254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:19.737254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:19.737254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:19.737254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:19.737254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:19.737254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:19.737254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:19.737254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:41:19.737254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:19.737254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 12:41:19.737254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:19.737254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 521/1023] 10.222.0.65 () {56 vars in 1105 bytes} [Wed Aug 3 12:41:19 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:19.743041 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1f961-9087-49d2-930a-9dccc1565517 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:19.792301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:19.856606 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1f961-9087-49d2-930a-9dccc1565517 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:19.897805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073891ab-8cb9-4d1d-8b7a-8df0615588f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:19.898423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073891ab-8cb9-4d1d-8b7a-8df0615588f0 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:19.898723 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073891ab-8cb9-4d1d-8b7a-8df0615588f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:19.898993 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073891ab-8cb9-4d1d-8b7a-8df0615588f0 None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:19.899688 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073891ab-8cb9-4d1d-8b7a-8df0615588f0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:19.899989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073891ab-8cb9-4d1d-8b7a-8df0615588f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:19.901343 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073891ab-8cb9-4d1d-8b7a-8df0615588f0 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:19.910558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 503/1024] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:19.912260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80de4cd-85d4-4526-affb-35e2bac2b69a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:20.017666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80de4cd-85d4-4526-affb-35e2bac2b69a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:20.051099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70943a08-9a1f-481d-be69-216c3ab7268a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:20.051779 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70943a08-9a1f-481d-be69-216c3ab7268a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:20.052118 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70943a08-9a1f-481d-be69-216c3ab7268a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:20.052397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70943a08-9a1f-481d-be69-216c3ab7268a None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:20.053072 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70943a08-9a1f-481d-be69-216c3ab7268a None admin] RBAC: Adding query filter params (type=image) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:20.053378 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70943a08-9a1f-481d-be69-216c3ab7268a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:20.054832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70943a08-9a1f-481d-be69-216c3ab7268a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:20.065444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 522/1025] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:20.066715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073891ab-8cb9-4d1d-8b7a-8df0615588f0 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:20.169329 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073891ab-8cb9-4d1d-8b7a-8df0615588f0 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:20.171443 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:20.205719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4153a18c-a2cb-45e6-8a9d-b6df60b841bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:20.206344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153a18c-a2cb-45e6-8a9d-b6df60b841bf None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:20.206654 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153a18c-a2cb-45e6-8a9d-b6df60b841bf None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:20.206927 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153a18c-a2cb-45e6-8a9d-b6df60b841bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:20.207678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153a18c-a2cb-45e6-8a9d-b6df60b841bf None admin] RBAC: Adding query filter params (interface=public, service_id=56f4923221c64e1aa506b0a2d865acf4, region_id=RegionOne) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:20.207989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153a18c-a2cb-45e6-8a9d-b6df60b841bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:20.209344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4153a18c-a2cb-45e6-8a9d-b6df60b841bf None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:20.219331 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 504/1026] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 3 12:41:20 2022] GET /identity/v3/endpoints?service_id=56f4923221c64e1aa506b0a2d865acf4&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:20.220575 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70943a08-9a1f-481d-be69-216c3ab7268a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:20.261075 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:20.323591 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70943a08-9a1f-481d-be69-216c3ab7268a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:20.357412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143ea82f-4e12-452b-a72f-860307e70c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:20.358595 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ea82f-4e12-452b-a72f-860307e70c6e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:20.358881 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ea82f-4e12-452b-a72f-860307e70c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:20.359151 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143ea82f-4e12-452b-a72f-860307e70c6e None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:20.359798 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ea82f-4e12-452b-a72f-860307e70c6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:20.361145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143ea82f-4e12-452b-a72f-860307e70c6e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:20.374760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 523/1027] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 3 12:41:20 2022] GET /identity/v3/services => generated 1957 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:41:20.679781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5156d-3c3d-4769-833e-b506c6210e46 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:20.680058 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5156d-3c3d-4769-833e-b506c6210e46 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:20.680252 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5156d-3c3d-4769-833e-b506c6210e46 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:20.681607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 505/1028] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:20.687145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cace3e-cbbc-4677-859a-1a36fc069076 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:20.687433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cace3e-cbbc-4677-859a-1a36fc069076 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:20.687714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cace3e-cbbc-4677-859a-1a36fc069076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:20.706766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-13cace3e-cbbc-4677-859a-1a36fc069076 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:20.707959 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-13cace3e-cbbc-4677-859a-1a36fc069076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:20.776359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 524/1029] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:20.799865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5156d-3c3d-4769-833e-b506c6210e46 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:20.911997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5156d-3c3d-4769-833e-b506c6210e46 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:20.944893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23deeaef-5e82-41e8-b355-dc5e6b90f21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:20.945464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23deeaef-5e82-41e8-b355-dc5e6b90f21f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:20.945742 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23deeaef-5e82-41e8-b355-dc5e6b90f21f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:20.946014 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23deeaef-5e82-41e8-b355-dc5e6b90f21f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:20.956982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23deeaef-5e82-41e8-b355-dc5e6b90f21f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:41:20.961834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-23deeaef-5e82-41e8-b355-dc5e6b90f21f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 12:41:20.961834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:20.961834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:20.961834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:20.961834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:20.961834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:20.961834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:20.961834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:20.961834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:20.961834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:20.961834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:20.961834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:20.961834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:41:20.961834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:41:20.961834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:41:20.961834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:41:20.963059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:41:20.963059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:41:20.963059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:41:20.963059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:41:20.963059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:20.963059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:20.963059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:20.963059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:20.963059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:20.963059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:20.963059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:20.963059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:20.963059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:20.963059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:20.963059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:20.963059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:20.963916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:20.963916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:20.963916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:20.963916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:20.963916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:41:20.963916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:41:20.963916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:41:20.963916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:41:20.963916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:41:20.963916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:41:20.963916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 12:41:20.963916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:20.966790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 506/1030] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 3 12:41:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:20.969696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cace3e-cbbc-4677-859a-1a36fc069076 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:20.999828 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:21.069468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cace3e-cbbc-4677-859a-1a36fc069076 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:21.103531 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c463810-47da-4107-bdab-fc879e32ca90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:21.104148 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c463810-47da-4107-bdab-fc879e32ca90 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:21.104424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c463810-47da-4107-bdab-fc879e32ca90 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:21.104691 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c463810-47da-4107-bdab-fc879e32ca90 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:21.105398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c463810-47da-4107-bdab-fc879e32ca90 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:21.105698 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c463810-47da-4107-bdab-fc879e32ca90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:21.107076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c463810-47da-4107-bdab-fc879e32ca90 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:21.119951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 525/1031] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 3 12:41:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:21.121351 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23deeaef-5e82-41e8-b355-dc5e6b90f21f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:21.151638 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:21.226413 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23deeaef-5e82-41e8-b355-dc5e6b90f21f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:21.260060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e881d-321e-4083-be79-58683e136e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:21.260700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e881d-321e-4083-be79-58683e136e6f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:21.260980 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e881d-321e-4083-be79-58683e136e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:21.261253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e881d-321e-4083-be79-58683e136e6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:21.272327 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e881d-321e-4083-be79-58683e136e6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:21.273757 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e881d-321e-4083-be79-58683e136e6f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:21.287790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-a84e881d-321e-4083-be79-58683e136e6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:21.287790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:21.287790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:21.287790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:21.287790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:21.287790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:21.287790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:21.287790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:21.287790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:21.287790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:21.287790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:21.287790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:21.287790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:41:21.287790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:41:21.287790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:21.288698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:21.294312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:41:21.294312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:41:21.294312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:21.294312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:21.294312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 507/1032] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:21.296272 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c463810-47da-4107-bdab-fc879e32ca90 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:21.392402 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:21.400037 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c463810-47da-4107-bdab-fc879e32ca90 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:21.436913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6018468-fd8f-4802-9c89-8677a43185e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:21.437664 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6018468-fd8f-4802-9c89-8677a43185e9 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:21.438045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6018468-fd8f-4802-9c89-8677a43185e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:21.438418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6018468-fd8f-4802-9c89-8677a43185e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:21.439233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6018468-fd8f-4802-9c89-8677a43185e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:21.439637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6018468-fd8f-4802-9c89-8677a43185e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:21.441089 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6018468-fd8f-4802-9c89-8677a43185e9 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:21.457752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 526/1033] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:21.459273 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e881d-321e-4083-be79-58683e136e6f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:21.541582 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:21.566387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e881d-321e-4083-be79-58683e136e6f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:21.600817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c528a0-3d8b-4efc-8b1f-f1c15a7a1bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:21.601666 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c528a0-3d8b-4efc-8b1f-f1c15a7a1bce None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:21.602093 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c528a0-3d8b-4efc-8b1f-f1c15a7a1bce None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:21.602682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c528a0-3d8b-4efc-8b1f-f1c15a7a1bce None admin] PATH_INFO: `/v3/users/nova` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:21.610978 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c528a0-3d8b-4efc-8b1f-f1c15a7a1bce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:21.612594 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c528a0-3d8b-4efc-8b1f-f1c15a7a1bce None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:21.624656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-56c528a0-3d8b-4efc-8b1f-f1c15a7a1bce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:41:21.624656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:21.624656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:21.624656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:21.624656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:21.624656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:21.624656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:21.624656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:21.624656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:21.624656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:21.624656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:21.624656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:21.624656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:41:21.624656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:41:21.624656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:41:21.624656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:21.625935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:21.627189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:21.627189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:21.627189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:41:21.627189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:41:21.627189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:41:21.627189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:41:21.627189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:41:21.627189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:41:21.627189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:41:21.627189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:21.627189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 508/1034] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 3 12:41:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:21.629591 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb6d6c-9f5f-4ea9-aa06-f7ebeb4ec1d1 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:21.629882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb6d6c-9f5f-4ea9-aa06-f7ebeb4ec1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:21.630160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb6d6c-9f5f-4ea9-aa06-f7ebeb4ec1d1 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:21.631635 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 527/1035] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:21.632617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c528a0-3d8b-4efc-8b1f-f1c15a7a1bce None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:21.636556 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5dabf-1f0a-442b-b948-5e028e1249e6 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:21.636841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5dabf-1f0a-442b-b948-5e028e1249e6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:21.637158 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5dabf-1f0a-442b-b948-5e028e1249e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:21.654387 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-95d5dabf-1f0a-442b-b948-5e028e1249e6 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:21.655605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-95d5dabf-1f0a-442b-b948-5e028e1249e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:21.724936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 528/1036] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 12:41:21.739968 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c528a0-3d8b-4efc-8b1f-f1c15a7a1bce None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:21.748907 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5dabf-1f0a-442b-b948-5e028e1249e6 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:21.776686 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a776c-1164-4fe9-896d-5d683a7d84b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:21.777564 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a776c-1164-4fe9-896d-5d683a7d84b1 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:21.778066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a776c-1164-4fe9-896d-5d683a7d84b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:21.778570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6a776c-1164-4fe9-896d-5d683a7d84b1 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:21.780023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a776c-1164-4fe9-896d-5d683a7d84b1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:21.780523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a776c-1164-4fe9-896d-5d683a7d84b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:21.782052 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6a776c-1164-4fe9-896d-5d683a7d84b1 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:21.791466 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:21.804245 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 509/1037] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 3 12:41:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:21.809813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a776c-1164-4fe9-896d-5d683a7d84b1 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:21.836141 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:21.854094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5dabf-1f0a-442b-b948-5e028e1249e6 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:21.892359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e8b3e-f09d-47bf-ac91-0ba5c3f69c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:21.893083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e8b3e-f09d-47bf-ac91-0ba5c3f69c76 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:21.893843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e8b3e-f09d-47bf-ac91-0ba5c3f69c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:21.894190 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e8b3e-f09d-47bf-ac91-0ba5c3f69c76 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:21.895438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e8b3e-f09d-47bf-ac91-0ba5c3f69c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:21.897508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47e8b3e-f09d-47bf-ac91-0ba5c3f69c76 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:21.900730 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 529/1038] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:21.905605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e8b3e-f09d-47bf-ac91-0ba5c3f69c76 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:21.924803 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6a776c-1164-4fe9-896d-5d683a7d84b1 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:21.960615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9751ec-93e3-4bcb-a611-be761827deff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:21.961341 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9751ec-93e3-4bcb-a611-be761827deff None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:21.961619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9751ec-93e3-4bcb-a611-be761827deff None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:21.962004 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9751ec-93e3-4bcb-a611-be761827deff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:22.011645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47e8b3e-f09d-47bf-ac91-0ba5c3f69c76 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:22.035808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:22.044736 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9751ec-93e3-4bcb-a611-be761827deff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:22.046967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9751ec-93e3-4bcb-a611-be761827deff None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:22.048901 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5e07b-fea1-4e7a-964d-a9a73fd6f3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:22.050188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5e07b-fea1-4e7a-964d-a9a73fd6f3af None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:22.050188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5e07b-fea1-4e7a-964d-a9a73fd6f3af None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:22.050188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5e07b-fea1-4e7a-964d-a9a73fd6f3af None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:22.050801 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5e07b-fea1-4e7a-964d-a9a73fd6f3af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:22.052195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5e07b-fea1-4e7a-964d-a9a73fd6f3af None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:22.086598 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 530/1039] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:41:21 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:41:22.160160 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 510/1040] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:41:21 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:41:22.165625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5e07b-fea1-4e7a-964d-a9a73fd6f3af None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:22.177540 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:22.280890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5e07b-fea1-4e7a-964d-a9a73fd6f3af None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:22.315622 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a1b9c0-ba50-4308-bda2-8fc8f8262671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:22.316275 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1b9c0-ba50-4308-bda2-8fc8f8262671 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:22.316555 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1b9c0-ba50-4308-bda2-8fc8f8262671 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:22.316827 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a1b9c0-ba50-4308-bda2-8fc8f8262671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:22.328622 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1b9c0-ba50-4308-bda2-8fc8f8262671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:22.330059 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a1b9c0-ba50-4308-bda2-8fc8f8262671 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:22.344922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-e9a1b9c0-ba50-4308-bda2-8fc8f8262671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:22.344922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:22.344922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:22.344922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:22.344922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:22.344922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:22.344922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:22.344922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:22.344922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:22.344922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:22.344922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:22.344922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:22.344922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:41:22.344922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:41:22.344922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:22.345848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:22.346790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:41:22.346790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:41:22.346790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:22.346790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:22.346790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 531/1041] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:22.351647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9751ec-93e3-4bcb-a611-be761827deff None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:22.392966 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19900f66-0414-4946-b257-d0ee8e0f66b7 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:22.393310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19900f66-0414-4946-b257-d0ee8e0f66b7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:22.393507 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19900f66-0414-4946-b257-d0ee8e0f66b7 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:22.395018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 532/1042] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:22.399642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ba200-94f6-49fa-937f-d07b93df7bd5 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:22.399929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ba200-94f6-49fa-937f-d07b93df7bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:22.400204 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ba200-94f6-49fa-937f-d07b93df7bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:22.418334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-eb8ba200-94f6-49fa-937f-d07b93df7bd5 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:22.419551 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-eb8ba200-94f6-49fa-937f-d07b93df7bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:22.449416 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:22.457515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9751ec-93e3-4bcb-a611-be761827deff None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:22.476011 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:22.493871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 533/1043] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 12:41:22.495770 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce70a4-5276-4815-a832-5723af26ce9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:22.496486 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce70a4-5276-4815-a832-5723af26ce9e None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:22.496838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce70a4-5276-4815-a832-5723af26ce9e None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:22.497182 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fce70a4-5276-4815-a832-5723af26ce9e None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:22.497946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce70a4-5276-4815-a832-5723af26ce9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:22.498881 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce70a4-5276-4815-a832-5723af26ce9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:22.500307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fce70a4-5276-4815-a832-5723af26ce9e None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:22.510461 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ba200-94f6-49fa-937f-d07b93df7bd5 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:22.516918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 511/1044] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:22.517858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce70a4-5276-4815-a832-5723af26ce9e None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:22.620900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ba200-94f6-49fa-937f-d07b93df7bd5 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:22.628096 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fce70a4-5276-4815-a832-5723af26ce9e None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:22.656375 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c31627-eaba-4254-a748-66234212ab7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:22.657014 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c31627-eaba-4254-a748-66234212ab7c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:22.657290 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c31627-eaba-4254-a748-66234212ab7c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:22.657557 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c31627-eaba-4254-a748-66234212ab7c None admin] PATH_INFO: `/v3/services/image` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:22.658176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c31627-eaba-4254-a748-66234212ab7c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:22.659566 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c31627-eaba-4254-a748-66234212ab7c None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:22.664128 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea32779-43a4-40ff-bcb9-64d73aa9ce60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:22.664771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea32779-43a4-40ff-bcb9-64d73aa9ce60 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:22.665051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea32779-43a4-40ff-bcb9-64d73aa9ce60 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:22.665348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea32779-43a4-40ff-bcb9-64d73aa9ce60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-b3c31627-eaba-4254-a748-66234212ab7c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:22.668403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:22.669418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:22.669418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:22.669418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:22.669418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:22.669418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:22.669418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:22.669418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:41:22.669418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:41:22.669418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:41:22.669418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:41:22.669418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:22.669418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:22.669418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:22.669418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:22.669418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:22.669418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:22.670996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:22.670996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:22.670996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:22.670996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:22.670996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:22.670996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:22.670996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:22.670996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:22.670996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:22.670996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:22.670996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:41:22.670996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:41:22.670996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 03 12:41:22.670996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:22.670996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 534/1045] 10.222.0.65 () {56 vars in 1105 bytes} [Wed Aug 3 12:41:22 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:22.673293 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea32779-43a4-40ff-bcb9-64d73aa9ce60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:22.675030 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea32779-43a4-40ff-bcb9-64d73aa9ce60 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:22.675444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c31627-eaba-4254-a748-66234212ab7c None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:22.687365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-5ea32779-43a4-40ff-bcb9-64d73aa9ce60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:41:22.687365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:22.687365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:22.687365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:22.687365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:22.687365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:22.687365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:22.687365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:22.687365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:22.687365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:22.687365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:22.687365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:22.687365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:41:22.687365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:41:22.687365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:22.688279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:22.689235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:41:22.689235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:41:22.689235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:41:22.689235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:41:22.689235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:41:22.689235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:41:22.689235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:41:22.689235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:22.690861 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 512/1046] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:41:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:22.693997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea32779-43a4-40ff-bcb9-64d73aa9ce60 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:22.713396 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:22.735656 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:22.782199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c31627-eaba-4254-a748-66234212ab7c None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:22.811604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea32779-43a4-40ff-bcb9-64d73aa9ce60 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:22.818447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95932c1e-79a9-4c0a-929d-1552c4215084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:22.819094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95932c1e-79a9-4c0a-929d-1552c4215084 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:22.819377 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95932c1e-79a9-4c0a-929d-1552c4215084 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:22.819653 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95932c1e-79a9-4c0a-929d-1552c4215084 None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:22.820453 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95932c1e-79a9-4c0a-929d-1552c4215084 None admin] RBAC: Adding query filter params (name=image) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:22.820766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95932c1e-79a9-4c0a-929d-1552c4215084 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:22.822254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95932c1e-79a9-4c0a-929d-1552c4215084 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:22.832690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 535/1047] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:22.833939 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95932c1e-79a9-4c0a-929d-1552c4215084 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:22.851888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaaf33-d142-4ab3-bad7-39efdd4c8540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:22.852503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaaf33-d142-4ab3-bad7-39efdd4c8540 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:22.852785 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaaf33-d142-4ab3-bad7-39efdd4c8540 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:22.853058 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccaaf33-d142-4ab3-bad7-39efdd4c8540 None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:22.853808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccaaf33-d142-4ab3-bad7-39efdd4c8540 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:22.854112 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccaaf33-d142-4ab3-bad7-39efdd4c8540 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:22.855573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccaaf33-d142-4ab3-bad7-39efdd4c8540 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:22.868453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 513/1048] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:41:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:22.879496 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaaf33-d142-4ab3-bad7-39efdd4c8540 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:22.949903 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95932c1e-79a9-4c0a-929d-1552c4215084 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:22.957079 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:22.967028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:22.986067 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccaaf33-d142-4ab3-bad7-39efdd4c8540 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:22.988014 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb8fe4-0dc7-45ed-a784-4365e74b05cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:22.988646 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb8fe4-0dc7-45ed-a784-4365e74b05cc None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:22.988921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb8fe4-0dc7-45ed-a784-4365e74b05cc None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:22.989953 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bb8fe4-0dc7-45ed-a784-4365e74b05cc None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:22.990681 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb8fe4-0dc7-45ed-a784-4365e74b05cc None admin] RBAC: Adding query filter params (type=image) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:22.990989 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb8fe4-0dc7-45ed-a784-4365e74b05cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:22.992353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bb8fe4-0dc7-45ed-a784-4365e74b05cc None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:22.998084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 536/1049] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:23.004501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb8fe4-0dc7-45ed-a784-4365e74b05cc None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:23.024151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1c1b8-b1ae-4af6-a5e7-c5d553998910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:23.025066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1c1b8-b1ae-4af6-a5e7-c5d553998910 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:23.025500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1c1b8-b1ae-4af6-a5e7-c5d553998910 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:23.025926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1c1b8-b1ae-4af6-a5e7-c5d553998910 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:23.027044 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1c1b8-b1ae-4af6-a5e7-c5d553998910 None admin] RBAC: Adding query filter params (role.id=b9b9695215b144d4b974ba1266bb18c1, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=7b634f567dca42dc95df175a4f36c1d1) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:23.027664 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1c1b8-b1ae-4af6-a5e7-c5d553998910 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:23.029250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1c1b8-b1ae-4af6-a5e7-c5d553998910 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:23.035448 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 514/1050] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:41:22 2022] GET /identity/v3/role_assignments?user.id=7b634f567dca42dc95df175a4f36c1d1&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=b9b9695215b144d4b974ba1266bb18c1 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:23.119435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bb8fe4-0dc7-45ed-a784-4365e74b05cc None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:23.156498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d568de8d-4dc1-4535-8097-cfedb94f41b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:23.157435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568de8d-4dc1-4535-8097-cfedb94f41b9 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:23.157941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568de8d-4dc1-4535-8097-cfedb94f41b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:23.158370 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d568de8d-4dc1-4535-8097-cfedb94f41b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:23.159364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568de8d-4dc1-4535-8097-cfedb94f41b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:23.160955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d568de8d-4dc1-4535-8097-cfedb94f41b9 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:23.184755 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 537/1051] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:41:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:41:24.163663 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e9d3e-eec3-48e4-8984-cdea18d1e884 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:24.164173 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e9d3e-eec3-48e4-8984-cdea18d1e884 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:24.164173 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e9d3e-eec3-48e4-8984-cdea18d1e884 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:24.165562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 515/1052] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:24.171403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fe6f2-85c0-4d3e-b87d-b8ecad508f77 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:24.171919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fe6f2-85c0-4d3e-b87d-b8ecad508f77 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:24.171919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42fe6f2-85c0-4d3e-b87d-b8ecad508f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:24.190436 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-c42fe6f2-85c0-4d3e-b87d-b8ecad508f77 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:24.191629 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c42fe6f2-85c0-4d3e-b87d-b8ecad508f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:24.222091 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:24.268464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 538/1053] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:24.290696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e9d3e-eec3-48e4-8984-cdea18d1e884 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:24.395342 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e9d3e-eec3-48e4-8984-cdea18d1e884 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:24.428842 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23af86-9732-4b19-9875-49de9b87ea26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:24.429417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23af86-9732-4b19-9875-49de9b87ea26 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:24.429697 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23af86-9732-4b19-9875-49de9b87ea26 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:24.429969 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23af86-9732-4b19-9875-49de9b87ea26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:24.438335 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:24.441799 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23af86-9732-4b19-9875-49de9b87ea26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:41:24.447374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-4b23af86-9732-4b19-9875-49de9b87ea26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:41:24.447374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:24.447374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:24.447374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:24.447374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:24.447374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:24.447374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:24.447374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:24.447374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:24.447374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:24.447374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:24.447374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:24.447374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:41:24.447374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:41:24.447374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:41:24.447374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:41:24.448509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:41:24.448509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:41:24.448509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:41:24.448509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:41:24.448509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:24.448509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:24.448509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:24.448509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:24.448509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:24.448509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:24.448509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:24.448509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:24.448509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:24.448509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:24.448509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:24.448509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:24.449676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:24.449676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:24.449676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:24.449676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:24.449676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:41:24.449676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:41:24.449676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:41:24.449676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:41:24.449676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:41:24.449676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:41:24.449676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:41:24.449676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:24.452292 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 516/1054] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:41:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:24.458075 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fe6f2-85c0-4d3e-b87d-b8ecad508f77 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:24.558295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42fe6f2-85c0-4d3e-b87d-b8ecad508f77 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:24.592563 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400359c7-fc46-4032-acf4-e557031c8491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:24.593270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400359c7-fc46-4032-acf4-e557031c8491 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:24.593612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400359c7-fc46-4032-acf4-e557031c8491 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:24.593893 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400359c7-fc46-4032-acf4-e557031c8491 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:24.594643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400359c7-fc46-4032-acf4-e557031c8491 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:24.594959 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400359c7-fc46-4032-acf4-e557031c8491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:24.596336 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400359c7-fc46-4032-acf4-e557031c8491 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:24.604778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 539/1055] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:41:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:24.610908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23af86-9732-4b19-9875-49de9b87ea26 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:24.711919 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23af86-9732-4b19-9875-49de9b87ea26 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:24.746687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb24fc1-df7e-4324-a990-63bd0f7d47a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:24.747326 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb24fc1-df7e-4324-a990-63bd0f7d47a1 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:24.747608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb24fc1-df7e-4324-a990-63bd0f7d47a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:24.747884 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb24fc1-df7e-4324-a990-63bd0f7d47a1 None admin] PATH_INFO: `/v3/users/12bb4db088504478bfe05b43fd7f4eee` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:24.761355 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb24fc1-df7e-4324-a990-63bd0f7d47a1 None admin] RBAC: Authorizing `identity:get_user(user_id=12bb4db088504478bfe05b43fd7f4eee)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:24.762833 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb24fc1-df7e-4324-a990-63bd0f7d47a1 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:24.764796 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 517/1056] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:41:24 2022] GET /identity/v3/users/12bb4db088504478bfe05b43fd7f4eee => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:24.770307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400359c7-fc46-4032-acf4-e557031c8491 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:24.800118 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:24.881508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400359c7-fc46-4032-acf4-e557031c8491 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:24.916774 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e6b24-ca68-4dba-af24-3a35dde7deab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:24.917404 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e6b24-ca68-4dba-af24-3a35dde7deab None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:24.917675 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e6b24-ca68-4dba-af24-3a35dde7deab None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:24.917948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e6b24-ca68-4dba-af24-3a35dde7deab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:24.986884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e6b24-ca68-4dba-af24-3a35dde7deab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:24.988241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e6b24-ca68-4dba-af24-3a35dde7deab None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:25.013944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:25.077135 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114a4d7-f163-4e8f-85bf-1dd19751aec8 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:25.077419 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114a4d7-f163-4e8f-85bf-1dd19751aec8 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:25.077690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3114a4d7-f163-4e8f-85bf-1dd19751aec8 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:25.079074 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 518/1057] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:25.083775 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b92b5-ed37-4c58-9f58-4eecfc4cedd7 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:25.084057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b92b5-ed37-4c58-9f58-4eecfc4cedd7 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:25.084330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b92b5-ed37-4c58-9f58-4eecfc4cedd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:25.094526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 540/1058] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:41:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 12:41:25.096442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e6b24-ca68-4dba-af24-3a35dde7deab None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:25.102916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-0c1b92b5-ed37-4c58-9f58-4eecfc4cedd7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:25.104109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-0c1b92b5-ed37-4c58-9f58-4eecfc4cedd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:25.158915 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:25.174171 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 519/1059] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 12:41:25.196143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b92b5-ed37-4c58-9f58-4eecfc4cedd7 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:25.201010 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e6b24-ca68-4dba-af24-3a35dde7deab None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:25.227352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:25.236670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a04a1-ba6c-4354-9a76-f1a8ea2c053d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:25.237358 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a04a1-ba6c-4354-9a76-f1a8ea2c053d None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:25.237576 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a04a1-ba6c-4354-9a76-f1a8ea2c053d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:25.237846 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a04a1-ba6c-4354-9a76-f1a8ea2c053d None admin] PATH_INFO: `/v3/projects/90b79586a53c48d7b4519b349cf13299` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:25.239030 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a04a1-ba6c-4354-9a76-f1a8ea2c053d None admin] RBAC: Authorizing `identity:get_project(project_id=90b79586a53c48d7b4519b349cf13299)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:25.240439 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a04a1-ba6c-4354-9a76-f1a8ea2c053d None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:25.242078 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 541/1060] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:41:25 2022] GET /identity/v3/projects/90b79586a53c48d7b4519b349cf13299 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:25.243201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b243de-73c0-4831-a714-aa444d688ff1 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:25.243477 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b243de-73c0-4831-a714-aa444d688ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:25.243746 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b243de-73c0-4831-a714-aa444d688ff1 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:25.244928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 542/1061] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:25.248041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b243de-73c0-4831-a714-aa444d688ff1 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:25.298808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b92b5-ed37-4c58-9f58-4eecfc4cedd7 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:25.332588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bcddb9-e0cd-4be9-9a7d-328fdf02e341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:25.333225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcddb9-e0cd-4be9-9a7d-328fdf02e341 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:25.333505 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcddb9-e0cd-4be9-9a7d-328fdf02e341 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:25.333783 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcddb9-e0cd-4be9-9a7d-328fdf02e341 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:25.344528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bcddb9-e0cd-4be9-9a7d-328fdf02e341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:41:25.346840 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b243de-73c0-4831-a714-aa444d688ff1 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:25.349439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-30bcddb9-e0cd-4be9-9a7d-328fdf02e341 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 12:41:25.349439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:25.349439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:25.349439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:25.349439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:25.349439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:25.349439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:25.349439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:25.349439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:25.349439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:25.349439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:25.349439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:25.349439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:41:25.349439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:41:25.349439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:41:25.349439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:41:25.350393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:41:25.350393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:41:25.350393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:41:25.350393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:41:25.350393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:25.350393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:25.350393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:25.350393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:25.350393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:25.350393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:25.350393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:25.350393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:25.350393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:25.350393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:25.350393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:25.350393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:25.351314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:25.351314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:25.351314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:25.351314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:25.351314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:41:25.351314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:41:25.351314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:41:25.351314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:41:25.351314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:41:25.351314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:41:25.351314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 12:41:25.351314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:25.351314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 520/1062] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 3 12:41:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:25.352369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40a93f-e3ec-487a-a7d8-e2cc8b85aeb4 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:25.352369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40a93f-e3ec-487a-a7d8-e2cc8b85aeb4 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:25.352369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40a93f-e3ec-487a-a7d8-e2cc8b85aeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:25.369887 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-0b40a93f-e3ec-487a-a7d8-e2cc8b85aeb4 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:25.371104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-0b40a93f-e3ec-487a-a7d8-e2cc8b85aeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:25.381242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ff358-10c9-4eb3-a3b0-eb54ea00c2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:25.381851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ff358-10c9-4eb3-a3b0-eb54ea00c2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:25.382132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ff358-10c9-4eb3-a3b0-eb54ea00c2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:25.382406 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ff358-10c9-4eb3-a3b0-eb54ea00c2f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:25.383218 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ff358-10c9-4eb3-a3b0-eb54ea00c2f8 None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=90b79586a53c48d7b4519b349cf13299, user.id=12bb4db088504478bfe05b43fd7f4eee) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:25.383527 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ff358-10c9-4eb3-a3b0-eb54ea00c2f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:25.384941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ff358-10c9-4eb3-a3b0-eb54ea00c2f8 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:25.390741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 543/1063] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:41:25 2022] GET /identity/v3/role_assignments?user.id=12bb4db088504478bfe05b43fd7f4eee&scope.project.id=90b79586a53c48d7b4519b349cf13299&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:25.391516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ff358-10c9-4eb3-a3b0-eb54ea00c2f8 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:25.419597 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:25.445676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 521/1064] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:25.468775 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40a93f-e3ec-487a-a7d8-e2cc8b85aeb4 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:25.474542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:25.522392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ff358-10c9-4eb3-a3b0-eb54ea00c2f8 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:25.563469 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc266f2b-3f52-44e5-9386-9db83b801b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:25.564098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc266f2b-3f52-44e5-9386-9db83b801b60 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:25.564382 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc266f2b-3f52-44e5-9386-9db83b801b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:25.564656 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc266f2b-3f52-44e5-9386-9db83b801b60 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:25.565373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc266f2b-3f52-44e5-9386-9db83b801b60 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:25.565676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc266f2b-3f52-44e5-9386-9db83b801b60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:25.567072 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc266f2b-3f52-44e5-9386-9db83b801b60 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:25.575580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 544/1065] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 3 12:41:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:25.582569 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc266f2b-3f52-44e5-9386-9db83b801b60 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:25.587679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40a93f-e3ec-487a-a7d8-e2cc8b85aeb4 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:25.623977 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21606c-31b1-44e0-9ff8-c0cb0c80f63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:25.625058 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21606c-31b1-44e0-9ff8-c0cb0c80f63a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:25.625058 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21606c-31b1-44e0-9ff8-c0cb0c80f63a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:25.625274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21606c-31b1-44e0-9ff8-c0cb0c80f63a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:25.675732 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:25.683428 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc266f2b-3f52-44e5-9386-9db83b801b60 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:25.702326 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21606c-31b1-44e0-9ff8-c0cb0c80f63a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:25.704089 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:25.705003 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21606c-31b1-44e0-9ff8-c0cb0c80f63a None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:25.719724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd18243-7e82-470f-adbb-83bc103052c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:25.720581 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd18243-7e82-470f-adbb-83bc103052c0 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:25.721030 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd18243-7e82-470f-adbb-83bc103052c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:25.721450 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd18243-7e82-470f-adbb-83bc103052c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:25.732256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd18243-7e82-470f-adbb-83bc103052c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:25.733888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd18243-7e82-470f-adbb-83bc103052c0 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:25.750131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-5cd18243-7e82-470f-adbb-83bc103052c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:25.750131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:25.750131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:25.750131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:25.750131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:25.750131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:25.750131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:25.750131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:25.750131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:25.750131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:25.750131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:25.750131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:25.750131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:41:25.750131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:41:25.750131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:25.751274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:25.752417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:41:25.752417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:41:25.752417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:25.752417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:25.758748 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 545/1066] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:25.759825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd18243-7e82-470f-adbb-83bc103052c0 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:25.811619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 522/1067] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:41:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:41:25.817630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21606c-31b1-44e0-9ff8-c0cb0c80f63a None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:25.880556 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd18243-7e82-470f-adbb-83bc103052c0 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:25.902424 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:25.918423 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcc005-46d4-4733-89e1-6c82d353f2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:25.919069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcc005-46d4-4733-89e1-6c82d353f2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:25.919350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcc005-46d4-4733-89e1-6c82d353f2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:25.919873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dcc005-46d4-4733-89e1-6c82d353f2e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:25.920350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcc005-46d4-4733-89e1-6c82d353f2e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:25.920655 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcc005-46d4-4733-89e1-6c82d353f2e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:25.922012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dcc005-46d4-4733-89e1-6c82d353f2e1 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:25.926463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21606c-31b1-44e0-9ff8-c0cb0c80f63a None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:25.934807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 546/1068] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:25.940433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcc005-46d4-4733-89e1-6c82d353f2e1 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:25.965929 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce5784-dc02-4dde-9673-410ba26d4c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:25.966971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce5784-dc02-4dde-9673-410ba26d4c6d None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:25.968007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce5784-dc02-4dde-9673-410ba26d4c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:25.968414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce5784-dc02-4dde-9673-410ba26d4c6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:25.974139 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:25.979275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce5784-dc02-4dde-9673-410ba26d4c6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:25.980724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce5784-dc02-4dde-9673-410ba26d4c6d None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:25.996209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-98ce5784-dc02-4dde-9673-410ba26d4c6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:25.996209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:25.996209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:25.996209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:25.996209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:25.996209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:25.996209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:25.996209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:25.996209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:25.996209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:25.996209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:25.996209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:25.996209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:41:25.996209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:41:25.996209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:25.997155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:25.998071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:41:25.998071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:41:25.998071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:25.998071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:25.998071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 523/1069] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:26.003552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce5784-dc02-4dde-9673-410ba26d4c6d None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:26.055282 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dcc005-46d4-4733-89e1-6c82d353f2e1 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:26.092216 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad2b01-07ab-413f-a3bc-9ad215888430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:26.092783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad2b01-07ab-413f-a3bc-9ad215888430 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:26.093067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad2b01-07ab-413f-a3bc-9ad215888430 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:26.093349 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad2b01-07ab-413f-a3bc-9ad215888430 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:26.100853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad2b01-07ab-413f-a3bc-9ad215888430 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:26.102281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad2b01-07ab-413f-a3bc-9ad215888430 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:26.114725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-d8ad2b01-07ab-413f-a3bc-9ad215888430 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:41:26.114725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:26.114725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:26.114725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:26.114725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:26.114725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:26.114725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:26.114725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:26.114725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:26.114725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:26.114725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:26.114725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:26.114725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:41:26.114725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:41:26.114725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:41:26.114725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:26.115711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:26.116666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:26.116666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:26.116666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:41:26.116666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:41:26.116666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:41:26.116666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:41:26.116666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:41:26.116666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:41:26.116666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:41:26.116666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:26.116666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 547/1070] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 3 12:41:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:26.121658 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad2b01-07ab-413f-a3bc-9ad215888430 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:26.127718 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce5784-dc02-4dde-9673-410ba26d4c6d None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:26.157215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:26.165629 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3a0a0-c9bf-4825-a85f-1f13f7f5d8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:26.166250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3a0a0-c9bf-4825-a85f-1f13f7f5d8eb None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:26.166559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3a0a0-c9bf-4825-a85f-1f13f7f5d8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:26.166840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3a0a0-c9bf-4825-a85f-1f13f7f5d8eb None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:26.167539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3a0a0-c9bf-4825-a85f-1f13f7f5d8eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:26.168254 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3a0a0-c9bf-4825-a85f-1f13f7f5d8eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:26.169971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3a0a0-c9bf-4825-a85f-1f13f7f5d8eb None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:26.183389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 524/1071] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:26.231914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:26.244082 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad2b01-07ab-413f-a3bc-9ad215888430 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:26.280908 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e919eb-173b-4548-a657-707660abc319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:26.281859 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e919eb-173b-4548-a657-707660abc319 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:26.282326 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e919eb-173b-4548-a657-707660abc319 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:26.282830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e919eb-173b-4548-a657-707660abc319 None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:26.283820 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e919eb-173b-4548-a657-707660abc319 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:26.284338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e919eb-173b-4548-a657-707660abc319 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:26.285966 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e919eb-173b-4548-a657-707660abc319 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:26.306063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 548/1072] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 3 12:41:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:26.312414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3a0a0-c9bf-4825-a85f-1f13f7f5d8eb None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:26.419843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3a0a0-c9bf-4825-a85f-1f13f7f5d8eb None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:26.457362 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd9c39-5321-4d58-a563-0106d1b96ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:26.458018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd9c39-5321-4d58-a563-0106d1b96ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:26.458297 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd9c39-5321-4d58-a563-0106d1b96ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:26.458602 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd9c39-5321-4d58-a563-0106d1b96ab0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:26.470235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd9c39-5321-4d58-a563-0106d1b96ab0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:26.471774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd9c39-5321-4d58-a563-0106d1b96ab0 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:26.489235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-0cfd9c39-5321-4d58-a563-0106d1b96ab0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:26.489235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:26.489235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:26.489235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:26.489235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:26.489235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:26.489235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:26.489235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:26.489235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:26.489235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:26.489235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:26.489235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:26.489235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:41:26.489235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:41:26.489235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:26.490392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:26.491503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:41:26.491503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:41:26.491503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:26.491503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:26.491503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 525/1073] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:26.496224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e919eb-173b-4548-a657-707660abc319 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:26.593790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e919eb-173b-4548-a657-707660abc319 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:26.626233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7d577-a0a1-477a-9fa9-7b011cce2f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:26.626890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7d577-a0a1-477a-9fa9-7b011cce2f6e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:26.627174 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7d577-a0a1-477a-9fa9-7b011cce2f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:26.627448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7d577-a0a1-477a-9fa9-7b011cce2f6e None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:26.628134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7d577-a0a1-477a-9fa9-7b011cce2f6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:26.628441 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7d577-a0a1-477a-9fa9-7b011cce2f6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:26.629793 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7d577-a0a1-477a-9fa9-7b011cce2f6e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:26.641205 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 549/1074] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:26.646841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd9c39-5321-4d58-a563-0106d1b96ab0 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:26.689158 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:26.745155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd9c39-5321-4d58-a563-0106d1b96ab0 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:26.777484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4e768-c840-4406-9314-d70cfaf375d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:26.778117 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4e768-c840-4406-9314-d70cfaf375d3 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:26.778400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4e768-c840-4406-9314-d70cfaf375d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:26.778704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4e768-c840-4406-9314-d70cfaf375d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:26.785756 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4e768-c840-4406-9314-d70cfaf375d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:26.787223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4e768-c840-4406-9314-d70cfaf375d3 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:26.798400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-b4c4e768-c840-4406-9314-d70cfaf375d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:41:26.798400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:26.798400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:26.798400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:26.798400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:26.798400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:26.798400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:26.798400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:26.798400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:26.798400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:26.798400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:26.798400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:26.798400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:41:26.798400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:41:26.798400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:26.799489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:26.800421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:41:26.800421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:41:26.800421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:41:26.800421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:41:26.800421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:41:26.800421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:41:26.800421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:41:26.800421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:26.800421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 526/1075] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:41:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:26.805506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7d577-a0a1-477a-9fa9-7b011cce2f6e None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:26.818299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:26.911524 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7d577-a0a1-477a-9fa9-7b011cce2f6e None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:26.943716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adba182-92f7-4a41-b43f-2e8d32168312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:26.944256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adba182-92f7-4a41-b43f-2e8d32168312 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:26.944537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adba182-92f7-4a41-b43f-2e8d32168312 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:26.944809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adba182-92f7-4a41-b43f-2e8d32168312 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:26.945584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adba182-92f7-4a41-b43f-2e8d32168312 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:26.945889 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adba182-92f7-4a41-b43f-2e8d32168312 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:26.947338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adba182-92f7-4a41-b43f-2e8d32168312 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:26.959466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 550/1076] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:41:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:26.965858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4e768-c840-4406-9314-d70cfaf375d3 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:27.062392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4e768-c840-4406-9314-d70cfaf375d3 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:27.063951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:27.096200 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa11a8-42c4-4725-b516-6cd1ba43c612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:27.096857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa11a8-42c4-4725-b516-6cd1ba43c612 None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:27.097132 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa11a8-42c4-4725-b516-6cd1ba43c612 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:27.097401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa11a8-42c4-4725-b516-6cd1ba43c612 None admin] PATH_INFO: `/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/7b634f567dca42dc95df175a4f36c1d1/roles/b9b9695215b144d4b974ba1266bb18c1` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:27.099581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa11a8-42c4-4725-b516-6cd1ba43c612 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0774035ad0f41c986771ef4a41c7d60, user_id=7b634f567dca42dc95df175a4f36c1d1, role_id=b9b9695215b144d4b974ba1266bb18c1)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:27.100996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa11a8-42c4-4725-b516-6cd1ba43c612 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:27.113214 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 527/1077] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:41:26 2022] PUT /identity/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/7b634f567dca42dc95df175a4f36c1d1/roles/b9b9695215b144d4b974ba1266bb18c1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:41:27.374290 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f379eb-ec28-45f0-b6f2-37d0eb684530 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:27.374849 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f379eb-ec28-45f0-b6f2-37d0eb684530 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:27.374849 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f379eb-ec28-45f0-b6f2-37d0eb684530 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:27.376196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 551/1078] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:27.381467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671686-3398-49a2-83a4-d46ead988beb None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:27.381863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671686-3398-49a2-83a4-d46ead988beb None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:27.382025 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671686-3398-49a2-83a4-d46ead988beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:27.399844 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-7e671686-3398-49a2-83a4-d46ead988beb None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:27.401025 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-7e671686-3398-49a2-83a4-d46ead988beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:27.465591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 528/1079] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:27.488130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f379eb-ec28-45f0-b6f2-37d0eb684530 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:27.557049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:27.588798 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f379eb-ec28-45f0-b6f2-37d0eb684530 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:27.621245 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9acb7-6f04-4b9a-b5d5-02042dbc1e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:27.621874 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9acb7-6f04-4b9a-b5d5-02042dbc1e76 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:27.622148 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9acb7-6f04-4b9a-b5d5-02042dbc1e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:27.622416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9acb7-6f04-4b9a-b5d5-02042dbc1e76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:27.632649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9acb7-6f04-4b9a-b5d5-02042dbc1e76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:41:27.637381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-fff9acb7-6f04-4b9a-b5d5-02042dbc1e76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:41:27.637381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:27.637381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:27.637381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:27.637381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:27.637381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:27.637381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:27.637381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:27.637381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:27.637381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:27.637381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:27.637381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:27.637381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:41:27.637381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:41:27.637381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:41:27.637381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:41:27.638371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:41:27.638371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:41:27.638371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:41:27.638371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:41:27.638371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:27.638371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:27.638371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:27.638371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:27.638371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:27.638371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:27.638371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:27.638371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:27.638371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:27.638371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:27.638371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:27.638371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:27.639637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:27.639637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:27.639637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:27.639637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:27.639637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:41:27.639637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:41:27.639637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:41:27.639637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:41:27.639637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:41:27.639637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:41:27.639637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:41:27.639637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:27.639637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 552/1080] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:41:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:27.644450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e671686-3398-49a2-83a4-d46ead988beb None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:27.731169 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:27.741803 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e671686-3398-49a2-83a4-d46ead988beb None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:27.774199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d412ca-9fc4-4b6c-bf79-36f7cf3f2a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:27.774844 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d412ca-9fc4-4b6c-bf79-36f7cf3f2a0d None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:27.775127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d412ca-9fc4-4b6c-bf79-36f7cf3f2a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:27.775398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d412ca-9fc4-4b6c-bf79-36f7cf3f2a0d None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:27.776110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d412ca-9fc4-4b6c-bf79-36f7cf3f2a0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:27.776414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d412ca-9fc4-4b6c-bf79-36f7cf3f2a0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:27.777769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d412ca-9fc4-4b6c-bf79-36f7cf3f2a0d None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:27.786651 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 529/1081] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:41:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:27.792381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9acb7-6f04-4b9a-b5d5-02042dbc1e76 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:27.890231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9acb7-6f04-4b9a-b5d5-02042dbc1e76 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:27.917371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:27.924206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c70a4-145d-479b-a036-d1a3f48334a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:27.924838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c70a4-145d-479b-a036-d1a3f48334a7 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:27.925115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c70a4-145d-479b-a036-d1a3f48334a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:27.925384 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c70a4-145d-479b-a036-d1a3f48334a7 None admin] PATH_INFO: `/v3/users/12bb4db088504478bfe05b43fd7f4eee` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:27.926547 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c70a4-145d-479b-a036-d1a3f48334a7 None admin] RBAC: Authorizing `identity:get_user(user_id=12bb4db088504478bfe05b43fd7f4eee)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:27.927967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861c70a4-145d-479b-a036-d1a3f48334a7 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:27.929543 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 553/1082] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:41:27 2022] GET /identity/v3/users/12bb4db088504478bfe05b43fd7f4eee => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:27.935614 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d412ca-9fc4-4b6c-bf79-36f7cf3f2a0d None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:28.031356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d412ca-9fc4-4b6c-bf79-36f7cf3f2a0d None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:28.065337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606eeef-ec33-4b6f-b9d6-2c1096e5706a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:28.065984 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606eeef-ec33-4b6f-b9d6-2c1096e5706a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:28.066266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606eeef-ec33-4b6f-b9d6-2c1096e5706a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:28.066573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606eeef-ec33-4b6f-b9d6-2c1096e5706a None admin] PATH_INFO: `/v3/projects/90b79586a53c48d7b4519b349cf13299` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:28.067746 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606eeef-ec33-4b6f-b9d6-2c1096e5706a None admin] RBAC: Authorizing `identity:get_project(project_id=90b79586a53c48d7b4519b349cf13299)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:28.069187 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606eeef-ec33-4b6f-b9d6-2c1096e5706a None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:28.070901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 530/1083] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:41:27 2022] GET /identity/v3/projects/90b79586a53c48d7b4519b349cf13299 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:28.077281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c70a4-145d-479b-a036-d1a3f48334a7 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:28.173509 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c70a4-145d-479b-a036-d1a3f48334a7 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:28.207704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3d449-b856-4b02-9a98-5dd9ee895bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:28.208370 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3d449-b856-4b02-9a98-5dd9ee895bf0 None admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:28.208645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3d449-b856-4b02-9a98-5dd9ee895bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:28.208913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3d449-b856-4b02-9a98-5dd9ee895bf0 None admin] PATH_INFO: `/v3/projects/90b79586a53c48d7b4519b349cf13299/users/12bb4db088504478bfe05b43fd7f4eee/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:28.210854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe3d449-b856-4b02-9a98-5dd9ee895bf0 None admin] RBAC: Authorizing `identity:create_grant(project_id=90b79586a53c48d7b4519b349cf13299, user_id=12bb4db088504478bfe05b43fd7f4eee, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:28.212263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe3d449-b856-4b02-9a98-5dd9ee895bf0 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:28.225900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 554/1084] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:41:28 2022] PUT /identity/v3/projects/90b79586a53c48d7b4519b349cf13299/users/12bb4db088504478bfe05b43fd7f4eee/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:41:29.074714 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b07cc-7040-421c-b9c5-652648497bb1 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:29.075256 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b07cc-7040-421c-b9c5-652648497bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:29.075256 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b07cc-7040-421c-b9c5-652648497bb1 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:29.076630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 531/1085] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:29.081584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c46781-6112-4e37-be6d-a56c2ad1876a None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:29.082007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c46781-6112-4e37-be6d-a56c2ad1876a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:29.082146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c46781-6112-4e37-be6d-a56c2ad1876a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:29.099826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-42c46781-6112-4e37-be6d-a56c2ad1876a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:29.101013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-42c46781-6112-4e37-be6d-a56c2ad1876a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:29.152519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:29.167045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 555/1086] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:29.190857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b07cc-7040-421c-b9c5-652648497bb1 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:29.215307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:29.293305 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b07cc-7040-421c-b9c5-652648497bb1 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:29.327062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f918ec-a390-42cc-837b-684c717e8050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:29.327702 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f918ec-a390-42cc-837b-684c717e8050 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:29.327982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f918ec-a390-42cc-837b-684c717e8050 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:29.328257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f918ec-a390-42cc-837b-684c717e8050 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:29.338603 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f918ec-a390-42cc-837b-684c717e8050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:41:29.344005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-01f918ec-a390-42cc-837b-684c717e8050 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 12:41:29.344005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:29.344005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:29.344005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:29.344005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:29.344005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:29.344005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:29.344005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:29.344005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:29.344005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:29.344005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:29.344005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:29.344005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:41:29.344005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:41:29.344005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:41:29.344005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:41:29.345110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:41:29.345110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:41:29.345110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:41:29.345110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:41:29.345110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:29.345110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:29.345110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:29.345110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:29.345110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:29.345110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:29.345110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:29.345110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:29.345110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:29.345110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:29.345110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:29.345110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:29.346005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:29.346005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:29.346005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:29.346005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:29.346005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:41:29.346005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:41:29.346005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:41:29.346005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:41:29.346005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:41:29.346005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:41:29.346005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 03 12:41:29.346005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:29.346005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 532/1087] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 3 12:41:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:29.351160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c46781-6112-4e37-be6d-a56c2ad1876a None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:29.447385 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c46781-6112-4e37-be6d-a56c2ad1876a None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:29.479952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb552ad-a0f9-48d4-93ff-1d6b85f3f0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:29.480560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb552ad-a0f9-48d4-93ff-1d6b85f3f0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:29.480841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb552ad-a0f9-48d4-93ff-1d6b85f3f0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:29.481115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb552ad-a0f9-48d4-93ff-1d6b85f3f0d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:29.481826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb552ad-a0f9-48d4-93ff-1d6b85f3f0d6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:29.482131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb552ad-a0f9-48d4-93ff-1d6b85f3f0d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:29.483555 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb552ad-a0f9-48d4-93ff-1d6b85f3f0d6 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:29.491238 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 556/1088] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 3 12:41:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:29.496799 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f918ec-a390-42cc-837b-684c717e8050 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:29.577719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:29.596002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f918ec-a390-42cc-837b-684c717e8050 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:29.630492 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e2261-1d1a-4a0f-aeec-251f19fe2eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:29.631144 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e2261-1d1a-4a0f-aeec-251f19fe2eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:29.631425 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e2261-1d1a-4a0f-aeec-251f19fe2eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:29.631700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e2261-1d1a-4a0f-aeec-251f19fe2eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:29.641930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e2261-1d1a-4a0f-aeec-251f19fe2eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:29.643400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e2261-1d1a-4a0f-aeec-251f19fe2eb8 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:29.657478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-f50e2261-1d1a-4a0f-aeec-251f19fe2eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:29.657478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:29.657478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:29.657478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:29.657478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:29.657478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:29.657478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:29.657478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:29.657478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:29.657478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:29.657478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:29.657478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:29.657478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:41:29.657478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:41:29.657478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:29.658390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:29.664069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:41:29.664069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:41:29.664069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:29.664069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:29.664069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 533/1089] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:29.664364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb552ad-a0f9-48d4-93ff-1d6b85f3f0d6 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:29.713901 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:29.763153 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb552ad-a0f9-48d4-93ff-1d6b85f3f0d6 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:29.795388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc14ee4-dd7c-4765-a636-3e1be125fb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:29.796005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc14ee4-dd7c-4765-a636-3e1be125fb97 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:29.796283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc14ee4-dd7c-4765-a636-3e1be125fb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:29.796551 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc14ee4-dd7c-4765-a636-3e1be125fb97 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:29.797227 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc14ee4-dd7c-4765-a636-3e1be125fb97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:29.797525 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc14ee4-dd7c-4765-a636-3e1be125fb97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:29.798907 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc14ee4-dd7c-4765-a636-3e1be125fb97 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:29.811296 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 557/1090] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:29.816687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e2261-1d1a-4a0f-aeec-251f19fe2eb8 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:29.919516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e2261-1d1a-4a0f-aeec-251f19fe2eb8 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:29.952489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cb2f2-a1fc-4be3-8322-a8469c9194f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:29.953122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cb2f2-a1fc-4be3-8322-a8469c9194f3 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:29.953398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cb2f2-a1fc-4be3-8322-a8469c9194f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:29.953665 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9cb2f2-a1fc-4be3-8322-a8469c9194f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:29.955560 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:29.961852 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9cb2f2-a1fc-4be3-8322-a8469c9194f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:29.963632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9cb2f2-a1fc-4be3-8322-a8469c9194f3 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:29.975520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-9e9cb2f2-a1fc-4be3-8322-a8469c9194f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:41:29.975520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:29.975520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:29.975520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:29.975520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:29.975520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:29.975520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:29.975520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:29.975520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:29.975520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:29.975520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:29.975520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:29.975520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:41:29.975520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:41:29.975520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:41:29.975520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:29.976681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:29.977677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:29.977677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:29.977677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:41:29.977677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:41:29.977677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:41:29.977677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:41:29.977677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:41:29.977677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:41:29.977677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 03 12:41:29.977677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:29.977677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 534/1091] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 3 12:41:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:29.982345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc14ee4-dd7c-4765-a636-3e1be125fb97 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:30.083493 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc14ee4-dd7c-4765-a636-3e1be125fb97 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:30.094645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:30.118409 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88535b9-eab7-491c-94bf-654e0b3a3513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:30.119049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88535b9-eab7-491c-94bf-654e0b3a3513 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:30.119324 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88535b9-eab7-491c-94bf-654e0b3a3513 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:30.119590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88535b9-eab7-491c-94bf-654e0b3a3513 None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:30.120368 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88535b9-eab7-491c-94bf-654e0b3a3513 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:30.120668 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88535b9-eab7-491c-94bf-654e0b3a3513 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:30.122064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88535b9-eab7-491c-94bf-654e0b3a3513 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:30.138426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 558/1092] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 3 12:41:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:30.144043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cb2f2-a1fc-4be3-8322-a8469c9194f3 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:30.168753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38feb4-958a-41ec-ae1f-59cb67753d52 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:30.169040 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38feb4-958a-41ec-ae1f-59cb67753d52 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:30.169317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38feb4-958a-41ec-ae1f-59cb67753d52 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:30.171266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 559/1093] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:41:30.175937 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd06ed-da06-4e31-9ac8-6a1b8cbb5712 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:30.176223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd06ed-da06-4e31-9ac8-6a1b8cbb5712 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:30.176494 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd06ed-da06-4e31-9ac8-6a1b8cbb5712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:30.193880 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-63cd06ed-da06-4e31-9ac8-6a1b8cbb5712 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:30.195093 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-63cd06ed-da06-4e31-9ac8-6a1b8cbb5712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:30.244111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9cb2f2-a1fc-4be3-8322-a8469c9194f3 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:30.259748 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 560/1094] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 12:41:30.275913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c8493-d0a5-443b-8695-272273d1eb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:30.276523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c8493-d0a5-443b-8695-272273d1eb8f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:30.276803 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c8493-d0a5-443b-8695-272273d1eb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:30.277075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c8493-d0a5-443b-8695-272273d1eb8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:30.282358 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd06ed-da06-4e31-9ac8-6a1b8cbb5712 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:30.337844 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:30.345608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c8493-d0a5-443b-8695-272273d1eb8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:30.347094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08c8493-d0a5-443b-8695-272273d1eb8f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:30.375627 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:30.379373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd06ed-da06-4e31-9ac8-6a1b8cbb5712 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:30.413021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e6c88-fa27-4df1-a716-1dc0c9bbc39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:30.413644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e6c88-fa27-4df1-a716-1dc0c9bbc39d None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:30.413928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e6c88-fa27-4df1-a716-1dc0c9bbc39d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:30.414196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e6c88-fa27-4df1-a716-1dc0c9bbc39d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:30.423903 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e6c88-fa27-4df1-a716-1dc0c9bbc39d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:41:30.428757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-eb4e6c88-fa27-4df1-a716-1dc0c9bbc39d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:41:30.428757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:30.428757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:30.428757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:30.428757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:30.428757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:30.428757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:30.428757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:30.428757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:30.428757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:30.428757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:30.428757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:30.428757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:41:30.428757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:41:30.428757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:41:30.428757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:41:30.429861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:41:30.429861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:41:30.429861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:41:30.429861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:41:30.429861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:30.429861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:30.429861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:30.429861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:30.429861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:30.429861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:30.429861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:30.429861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:30.429861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:30.429861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:30.429861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:30.429861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:30.430997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:30.430997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:30.430997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:30.430997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:30.430997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:41:30.430997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:41:30.430997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:41:30.430997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:41:30.430997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:41:30.430997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:41:30.430997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:41:30.430997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:30.434819 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 561/1095] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:41:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:30.436183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e6c88-fa27-4df1-a716-1dc0c9bbc39d None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:30.443949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 535/1096] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:41:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:41:30.449466 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c8493-d0a5-443b-8695-272273d1eb8f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:30.533019 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e6c88-fa27-4df1-a716-1dc0c9bbc39d None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:30.539717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:30.546771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c8493-d0a5-443b-8695-272273d1eb8f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:30.567230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026cb8b-2d89-4cbe-b565-8813deb8a91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:30.567840 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026cb8b-2d89-4cbe-b565-8813deb8a91e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:30.568119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026cb8b-2d89-4cbe-b565-8813deb8a91e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:30.568392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026cb8b-2d89-4cbe-b565-8813deb8a91e None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:30.569122 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026cb8b-2d89-4cbe-b565-8813deb8a91e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:30.569424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026cb8b-2d89-4cbe-b565-8813deb8a91e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:30.570809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026cb8b-2d89-4cbe-b565-8813deb8a91e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:30.579093 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 562/1097] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:41:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:30.581451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cad15b-a873-4a49-ba47-7eac843efab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:30.582091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cad15b-a873-4a49-ba47-7eac843efab1 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:30.582371 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cad15b-a873-4a49-ba47-7eac843efab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:30.582688 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cad15b-a873-4a49-ba47-7eac843efab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:30.584634 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026cb8b-2d89-4cbe-b565-8813deb8a91e None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:30.593544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cad15b-a873-4a49-ba47-7eac843efab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:30.595018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cad15b-a873-4a49-ba47-7eac843efab1 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:30.605563 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:30.609790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-38cad15b-a873-4a49-ba47-7eac843efab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:30.609790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:30.609790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:30.609790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:30.609790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:30.609790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:30.609790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:30.609790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:30.609790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:30.609790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:30.609790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:30.609790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:30.609790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:41:30.609790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:41:30.609790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:30.610816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:30.611728 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:41:30.611728 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:41:30.611728 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:41:30.611728 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:30.615679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 536/1098] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:30.616400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cad15b-a873-4a49-ba47-7eac843efab1 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:30.681553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026cb8b-2d89-4cbe-b565-8813deb8a91e None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:30.713489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cad15b-a873-4a49-ba47-7eac843efab1 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:30.715397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76180adb-27a9-4168-b982-eeb30c22c119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:30.716041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76180adb-27a9-4168-b982-eeb30c22c119 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:30.716323 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76180adb-27a9-4168-b982-eeb30c22c119 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:30.716602 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76180adb-27a9-4168-b982-eeb30c22c119 None admin] PATH_INFO: `/v3/users/12bb4db088504478bfe05b43fd7f4eee` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:30.718391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76180adb-27a9-4168-b982-eeb30c22c119 None admin] RBAC: Authorizing `identity:get_user(user_id=12bb4db088504478bfe05b43fd7f4eee)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:30.719855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76180adb-27a9-4168-b982-eeb30c22c119 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:30.721455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 563/1099] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:41:30 2022] GET /identity/v3/users/12bb4db088504478bfe05b43fd7f4eee => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:30.727064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76180adb-27a9-4168-b982-eeb30c22c119 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:30.751131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede3ce7-7a12-45ef-9e85-2b0b7ab30d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:30.751772 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede3ce7-7a12-45ef-9e85-2b0b7ab30d20 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:30.752054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede3ce7-7a12-45ef-9e85-2b0b7ab30d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:30.752334 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede3ce7-7a12-45ef-9e85-2b0b7ab30d20 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:30.753026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede3ce7-7a12-45ef-9e85-2b0b7ab30d20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:30.753330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede3ce7-7a12-45ef-9e85-2b0b7ab30d20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:30.754721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede3ce7-7a12-45ef-9e85-2b0b7ab30d20 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:30.766740 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 537/1100] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:30.772211 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede3ce7-7a12-45ef-9e85-2b0b7ab30d20 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:30.780168 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:30.813244 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:30.825415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76180adb-27a9-4168-b982-eeb30c22c119 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:30.858407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6cc5d5-658c-45af-bfff-d0c01e3820ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:30.859050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cc5d5-658c-45af-bfff-d0c01e3820ae None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:30.859329 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cc5d5-658c-45af-bfff-d0c01e3820ae None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:30.859630 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cc5d5-658c-45af-bfff-d0c01e3820ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:30.869961 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede3ce7-7a12-45ef-9e85-2b0b7ab30d20 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:30.905789 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546ab23-3921-4f6c-b1fe-49746d33a97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:30.906429 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546ab23-3921-4f6c-b1fe-49746d33a97f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:30.906742 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546ab23-3921-4f6c-b1fe-49746d33a97f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:30.907019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546ab23-3921-4f6c-b1fe-49746d33a97f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:30.913631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546ab23-3921-4f6c-b1fe-49746d33a97f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:30.915101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546ab23-3921-4f6c-b1fe-49746d33a97f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:30.926219 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cc5d5-658c-45af-bfff-d0c01e3820ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:30.926455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-3546ab23-3921-4f6c-b1fe-49746d33a97f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:41:30.926455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:30.926455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:30.926455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:30.926455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:30.926455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:30.926455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:30.926455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:30.926455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:30.926455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:30.926455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:30.926455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:30.926455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:41:30.926455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:41:30.926455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:30.927549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:30.928484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:41:30.928484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:41:30.928484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:41:30.928484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:41:30.928484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:41:30.928484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:41:30.928484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:41:30.928484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:30.928931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cc5d5-658c-45af-bfff-d0c01e3820ae None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:30.929011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 538/1101] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:41:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:41:30.933347 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546ab23-3921-4f6c-b1fe-49746d33a97f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:30.997649 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:31.012514 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:31.026282 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 564/1102] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:41:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 03 12:41:31.032239 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6cc5d5-658c-45af-bfff-d0c01e3820ae None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:31.033839 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546ab23-3921-4f6c-b1fe-49746d33a97f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:31.066642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a59923-9d80-401c-9abc-c6fabdb61fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:31.067176 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a59923-9d80-401c-9abc-c6fabdb61fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:31.067452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a59923-9d80-401c-9abc-c6fabdb61fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:31.067720 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a59923-9d80-401c-9abc-c6fabdb61fa6 None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:31.068458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a59923-9d80-401c-9abc-c6fabdb61fa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:31.068761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a59923-9d80-401c-9abc-c6fabdb61fa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:31.070166 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a59923-9d80-401c-9abc-c6fabdb61fa6 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:31.082376 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 539/1103] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:41:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:31.088259 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a59923-9d80-401c-9abc-c6fabdb61fa6 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:31.130581 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6cc5d5-658c-45af-bfff-d0c01e3820ae None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:31.163250 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06428949-3342-480d-b2ee-60235e6f72d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:31.163884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06428949-3342-480d-b2ee-60235e6f72d0 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:31.164218 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06428949-3342-480d-b2ee-60235e6f72d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:31.164492 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06428949-3342-480d-b2ee-60235e6f72d0 None admin] PATH_INFO: `/v3/projects/90b79586a53c48d7b4519b349cf13299` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:31.165633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06428949-3342-480d-b2ee-60235e6f72d0 None admin] RBAC: Authorizing `identity:get_project(project_id=90b79586a53c48d7b4519b349cf13299)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:31.167087 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06428949-3342-480d-b2ee-60235e6f72d0 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:31.168736 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 565/1104] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:41:31 2022] GET /identity/v3/projects/90b79586a53c48d7b4519b349cf13299 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:31.175278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06428949-3342-480d-b2ee-60235e6f72d0 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:31.187538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a59923-9d80-401c-9abc-c6fabdb61fa6 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:31.212378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:31.221758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05de3a-dd00-47e7-941f-cd5f9c567276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:31.222371 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05de3a-dd00-47e7-941f-cd5f9c567276 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:31.222704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05de3a-dd00-47e7-941f-cd5f9c567276 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:31.222980 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05de3a-dd00-47e7-941f-cd5f9c567276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:31.223780 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05de3a-dd00-47e7-941f-cd5f9c567276 None admin] RBAC: Adding query filter params (role.id=b9b9695215b144d4b974ba1266bb18c1, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=7b634f567dca42dc95df175a4f36c1d1) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:31.224093 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05de3a-dd00-47e7-941f-cd5f9c567276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:31.225519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05de3a-dd00-47e7-941f-cd5f9c567276 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:31.229644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:31.231816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 540/1105] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:41:31 2022] GET /identity/v3/role_assignments?user.id=7b634f567dca42dc95df175a4f36c1d1&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=b9b9695215b144d4b974ba1266bb18c1 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:31.276583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06428949-3342-480d-b2ee-60235e6f72d0 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:31.313190 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7031d45c-a17b-4056-a5ad-9fe7bf81638c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:31.313821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031d45c-a17b-4056-a5ad-9fe7bf81638c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:31.314099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031d45c-a17b-4056-a5ad-9fe7bf81638c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:31.314372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7031d45c-a17b-4056-a5ad-9fe7bf81638c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:31.315202 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031d45c-a17b-4056-a5ad-9fe7bf81638c None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=90b79586a53c48d7b4519b349cf13299, user.id=12bb4db088504478bfe05b43fd7f4eee) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:31.315512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031d45c-a17b-4056-a5ad-9fe7bf81638c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:31.317009 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7031d45c-a17b-4056-a5ad-9fe7bf81638c None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:31.323984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 566/1106] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:41:31 2022] GET /identity/v3/role_assignments?user.id=12bb4db088504478bfe05b43fd7f4eee&scope.project.id=90b79586a53c48d7b4519b349cf13299&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:33.339200 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfcd9b-98f5-44e3-8059-768a412766fb None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:33.339753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfcd9b-98f5-44e3-8059-768a412766fb None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:33.339753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfcd9b-98f5-44e3-8059-768a412766fb None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:33.341102 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 541/1107] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:33.346360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b10afb-375c-4bf3-821e-af8af61eafa6 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:33.346757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b10afb-375c-4bf3-821e-af8af61eafa6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:33.346950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b10afb-375c-4bf3-821e-af8af61eafa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:33.365003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-79b10afb-375c-4bf3-821e-af8af61eafa6 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:33.366176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-79b10afb-375c-4bf3-821e-af8af61eafa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:33.429658 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 567/1108] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:33.454301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfcd9b-98f5-44e3-8059-768a412766fb None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:33.550691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfcd9b-98f5-44e3-8059-768a412766fb None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:33.583606 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c5309-5f3d-48dd-8eb4-d6ae16d8a6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:33.584225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c5309-5f3d-48dd-8eb4-d6ae16d8a6ab None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:33.584500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c5309-5f3d-48dd-8eb4-d6ae16d8a6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:33.584768 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c5309-5f3d-48dd-8eb4-d6ae16d8a6ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:33.585905 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c5309-5f3d-48dd-8eb4-d6ae16d8a6ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:33.587345 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c5309-5f3d-48dd-8eb4-d6ae16d8a6ab None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:33.588910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 542/1109] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:33.594509 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b10afb-375c-4bf3-821e-af8af61eafa6 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:33.642240 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:33.691352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b10afb-375c-4bf3-821e-af8af61eafa6 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:33.723946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aac066-b2f3-4f43-a7a1-d30c194b2738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:33.724543 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aac066-b2f3-4f43-a7a1-d30c194b2738 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:33.724821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aac066-b2f3-4f43-a7a1-d30c194b2738 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:33.725088 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aac066-b2f3-4f43-a7a1-d30c194b2738 None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:33.725758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aac066-b2f3-4f43-a7a1-d30c194b2738 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:33.727197 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3aac066-b2f3-4f43-a7a1-d30c194b2738 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:33.748149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 568/1110] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:41:33 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:41:35.757651 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5c832-94ab-4183-bb2c-1d1d11a823fb None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:35.758221 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5c832-94ab-4183-bb2c-1d1d11a823fb None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:35.758221 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5c832-94ab-4183-bb2c-1d1d11a823fb None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:35.759578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 543/1111] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:35.765200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9617d47-dd6e-4b26-824d-42f5f7312e44 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:35.765616 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9617d47-dd6e-4b26-824d-42f5f7312e44 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:35.765753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9617d47-dd6e-4b26-824d-42f5f7312e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:35.784097 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-a9617d47-dd6e-4b26-824d-42f5f7312e44 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:35.785273 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a9617d47-dd6e-4b26-824d-42f5f7312e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:35.848718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 569/1112] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:35.871308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5c832-94ab-4183-bb2c-1d1d11a823fb None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:35.929205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:35.970986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5c832-94ab-4183-bb2c-1d1d11a823fb None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:36.003628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15c78d-31de-47f7-8ebd-b0a622e59a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:36.004247 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15c78d-31de-47f7-8ebd-b0a622e59a0f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:36.004523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15c78d-31de-47f7-8ebd-b0a622e59a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:36.004790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d15c78d-31de-47f7-8ebd-b0a622e59a0f None admin] PATH_INFO: `/v3/roles/79ff845a41e241229433560e2b0d6fe5` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:36.006207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15c78d-31de-47f7-8ebd-b0a622e59a0f None admin] RBAC: Authorizing `identity:get_role(role_id=79ff845a41e241229433560e2b0d6fe5)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:36.007587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d15c78d-31de-47f7-8ebd-b0a622e59a0f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:36.008948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 544/1113] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:41:35 2022] GET /identity/v3/roles/79ff845a41e241229433560e2b0d6fe5 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:36.015206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9617d47-dd6e-4b26-824d-42f5f7312e44 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:36.047357 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:36.111178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9617d47-dd6e-4b26-824d-42f5f7312e44 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:36.142742 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c6fb1-a50d-4d2a-9b89-af7f5719eba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:36.143365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c6fb1-a50d-4d2a-9b89-af7f5719eba7 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:36.143641 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c6fb1-a50d-4d2a-9b89-af7f5719eba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:36.143907 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c6fb1-a50d-4d2a-9b89-af7f5719eba7 None admin] PATH_INFO: `/v3/users/1a1b079ad62f4d7283f7577e1d9b294a` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:36.156720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c6fb1-a50d-4d2a-9b89-af7f5719eba7 None admin] RBAC: Authorizing `identity:get_user(user_id=1a1b079ad62f4d7283f7577e1d9b294a)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:36.158135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c6fb1-a50d-4d2a-9b89-af7f5719eba7 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:36.159752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 570/1114] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:41:36 2022] GET /identity/v3/users/1a1b079ad62f4d7283f7577e1d9b294a => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:36.165467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15c78d-31de-47f7-8ebd-b0a622e59a0f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:36.261147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d15c78d-31de-47f7-8ebd-b0a622e59a0f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:36.285335 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:36.293999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c0ecb-7f7a-4dba-bde4-0d6132b364f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:36.294658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c0ecb-7f7a-4dba-bde4-0d6132b364f3 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:36.295502 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c0ecb-7f7a-4dba-bde4-0d6132b364f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:36.295772 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c0ecb-7f7a-4dba-bde4-0d6132b364f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:36.360313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c0ecb-7f7a-4dba-bde4-0d6132b364f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:36.361733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454c0ecb-7f7a-4dba-bde4-0d6132b364f3 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:36.455594 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 545/1115] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:41:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:41:36.461309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c6fb1-a50d-4d2a-9b89-af7f5719eba7 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:36.550162 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:36.557523 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c6fb1-a50d-4d2a-9b89-af7f5719eba7 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:36.590393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07199ab6-4226-4395-b5b0-1f0b1b9d71f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:36.591044 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07199ab6-4226-4395-b5b0-1f0b1b9d71f2 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:36.591321 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07199ab6-4226-4395-b5b0-1f0b1b9d71f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:36.591590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07199ab6-4226-4395-b5b0-1f0b1b9d71f2 None admin] PATH_INFO: `/v3/projects/90b79586a53c48d7b4519b349cf13299` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:36.592718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07199ab6-4226-4395-b5b0-1f0b1b9d71f2 None admin] RBAC: Authorizing `identity:get_project(project_id=90b79586a53c48d7b4519b349cf13299)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:36.594121 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07199ab6-4226-4395-b5b0-1f0b1b9d71f2 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:36.595787 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 571/1116] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:41:36 2022] GET /identity/v3/projects/90b79586a53c48d7b4519b349cf13299 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:36.601748 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c0ecb-7f7a-4dba-bde4-0d6132b364f3 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:36.628377 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:36.698063 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c0ecb-7f7a-4dba-bde4-0d6132b364f3 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:36.730671 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc33cc-e2cb-40a2-9ba0-ee5e81ec3a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:36.731264 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc33cc-e2cb-40a2-9ba0-ee5e81ec3a8d None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:36.731544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc33cc-e2cb-40a2-9ba0-ee5e81ec3a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:36.731813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc33cc-e2cb-40a2-9ba0-ee5e81ec3a8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:36.733177 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc33cc-e2cb-40a2-9ba0-ee5e81ec3a8d None admin] RBAC: Adding query filter params (role.id=79ff845a41e241229433560e2b0d6fe5, scope.project.id=90b79586a53c48d7b4519b349cf13299, user.id=1a1b079ad62f4d7283f7577e1d9b294a) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:36.733480 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc33cc-e2cb-40a2-9ba0-ee5e81ec3a8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:36.734914 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc33cc-e2cb-40a2-9ba0-ee5e81ec3a8d None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:36.740111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 546/1117] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:41:36 2022] GET /identity/v3/role_assignments?user.id=1a1b079ad62f4d7283f7577e1d9b294a&scope.project.id=90b79586a53c48d7b4519b349cf13299&role.id=79ff845a41e241229433560e2b0d6fe5 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:38.687615 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c28e5e-12b1-414a-a118-dd5aa3921020 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:38.688167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c28e5e-12b1-414a-a118-dd5aa3921020 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:38.688167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c28e5e-12b1-414a-a118-dd5aa3921020 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:38.689499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 572/1118] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:38.694679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e373e-b5ee-4437-a9b9-4791495df0c9 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:38.695082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e373e-b5ee-4437-a9b9-4791495df0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:38.695229 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e373e-b5ee-4437-a9b9-4791495df0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:38.713154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-c17e373e-b5ee-4437-a9b9-4791495df0c9 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:38.714335 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c17e373e-b5ee-4437-a9b9-4791495df0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:38.780026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 547/1119] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:38.801505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c28e5e-12b1-414a-a118-dd5aa3921020 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:38.898813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c28e5e-12b1-414a-a118-dd5aa3921020 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:38.931163 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69844f-cc80-4111-aaad-6b701b56e654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:38.931789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69844f-cc80-4111-aaad-6b701b56e654 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:38.932067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69844f-cc80-4111-aaad-6b701b56e654 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:38.932337 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69844f-cc80-4111-aaad-6b701b56e654 None admin] PATH_INFO: `/v3/roles/79ff845a41e241229433560e2b0d6fe5` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:38.934004 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69844f-cc80-4111-aaad-6b701b56e654 None admin] RBAC: Authorizing `identity:get_role(role_id=79ff845a41e241229433560e2b0d6fe5)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:38.935371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69844f-cc80-4111-aaad-6b701b56e654 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:38.936681 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 573/1120] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:41:38 2022] GET /identity/v3/roles/79ff845a41e241229433560e2b0d6fe5 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:38.942450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e373e-b5ee-4437-a9b9-4791495df0c9 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:38.963495 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:39.043106 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e373e-b5ee-4437-a9b9-4791495df0c9 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:39.075554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6005a6-c8bc-4934-a584-38953609d98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:39.076780 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6005a6-c8bc-4934-a584-38953609d98a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:39.077057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6005a6-c8bc-4934-a584-38953609d98a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:39.077325 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6005a6-c8bc-4934-a584-38953609d98a None admin] PATH_INFO: `/v3/users/1a1b079ad62f4d7283f7577e1d9b294a` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:39.078473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6005a6-c8bc-4934-a584-38953609d98a None admin] RBAC: Authorizing `identity:get_user(user_id=1a1b079ad62f4d7283f7577e1d9b294a)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:39.079903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6005a6-c8bc-4934-a584-38953609d98a None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:39.081478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 548/1121] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:41:38 2022] GET /identity/v3/users/1a1b079ad62f4d7283f7577e1d9b294a => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:39.087109 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69844f-cc80-4111-aaad-6b701b56e654 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:39.116567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:39.182948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69844f-cc80-4111-aaad-6b701b56e654 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:39.215242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a0861-a091-46b8-9cb9-40179e0c6c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:39.215860 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a0861-a091-46b8-9cb9-40179e0c6c59 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:39.216134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a0861-a091-46b8-9cb9-40179e0c6c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:39.216681 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a0861-a091-46b8-9cb9-40179e0c6c59 None admin] PATH_INFO: `/v3/projects/90b79586a53c48d7b4519b349cf13299` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:39.217815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a0861-a091-46b8-9cb9-40179e0c6c59 None admin] RBAC: Authorizing `identity:get_project(project_id=90b79586a53c48d7b4519b349cf13299)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:39.219249 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a0861-a091-46b8-9cb9-40179e0c6c59 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:39.220855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 574/1122] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:41:39 2022] GET /identity/v3/projects/90b79586a53c48d7b4519b349cf13299 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:39.226595 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6005a6-c8bc-4934-a584-38953609d98a None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:39.310162 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:39.323220 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6005a6-c8bc-4934-a584-38953609d98a None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:39.355565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47de9854-5c93-45d2-9c3e-077b4bdf5d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:39.356212 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de9854-5c93-45d2-9c3e-077b4bdf5d38 None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:39.356491 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de9854-5c93-45d2-9c3e-077b4bdf5d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:39.356779 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47de9854-5c93-45d2-9c3e-077b4bdf5d38 None admin] PATH_INFO: `/v3/projects/90b79586a53c48d7b4519b349cf13299/users/1a1b079ad62f4d7283f7577e1d9b294a/roles/79ff845a41e241229433560e2b0d6fe5` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:39.359482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de9854-5c93-45d2-9c3e-077b4bdf5d38 None admin] RBAC: Authorizing `identity:create_grant(project_id=90b79586a53c48d7b4519b349cf13299, user_id=1a1b079ad62f4d7283f7577e1d9b294a, role_id=79ff845a41e241229433560e2b0d6fe5)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:39.360881 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47de9854-5c93-45d2-9c3e-077b4bdf5d38 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:39.372758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 549/1123] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:41:39 2022] PUT /identity/v3/projects/90b79586a53c48d7b4519b349cf13299/users/1a1b079ad62f4d7283f7577e1d9b294a/roles/79ff845a41e241229433560e2b0d6fe5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:41:41.312905 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daaefdf-c6d7-4ca9-bc0b-dbfedfdb8bcc None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:41.313541 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daaefdf-c6d7-4ca9-bc0b-dbfedfdb8bcc None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:41.313541 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daaefdf-c6d7-4ca9-bc0b-dbfedfdb8bcc None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:41.314868 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 575/1124] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:41.319919 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd60b1-323b-4a71-8171-e0c8b117d1fa None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:41.320321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd60b1-323b-4a71-8171-e0c8b117d1fa None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:41.320469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cd60b1-323b-4a71-8171-e0c8b117d1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:41.338150 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-f7cd60b1-323b-4a71-8171-e0c8b117d1fa None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:41.339366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f7cd60b1-323b-4a71-8171-e0c8b117d1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:41.403003 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 550/1125] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:41.426688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daaefdf-c6d7-4ca9-bc0b-dbfedfdb8bcc None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:41.520314 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:41.525232 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daaefdf-c6d7-4ca9-bc0b-dbfedfdb8bcc None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:41.558476 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cba86-7195-403c-b759-06bfe1889931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:41.559419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cba86-7195-403c-b759-06bfe1889931 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:41.559700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cba86-7195-403c-b759-06bfe1889931 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:41.559972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693cba86-7195-403c-b759-06bfe1889931 None admin] PATH_INFO: `/v3/roles/79ff845a41e241229433560e2b0d6fe5` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:41.561408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cba86-7195-403c-b759-06bfe1889931 None admin] RBAC: Authorizing `identity:get_role(role_id=79ff845a41e241229433560e2b0d6fe5)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:41.562784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693cba86-7195-403c-b759-06bfe1889931 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:41.564161 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 576/1126] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:41:41 2022] GET /identity/v3/roles/79ff845a41e241229433560e2b0d6fe5 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:41.569737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd60b1-323b-4a71-8171-e0c8b117d1fa None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:41.644966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:41.666696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cd60b1-323b-4a71-8171-e0c8b117d1fa None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:41.698968 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70884f01-253e-4a57-9c24-aef4b9e52f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:41.699590 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70884f01-253e-4a57-9c24-aef4b9e52f72 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:41.699868 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70884f01-253e-4a57-9c24-aef4b9e52f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:41.700138 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70884f01-253e-4a57-9c24-aef4b9e52f72 None admin] PATH_INFO: `/v3/users/1a1b079ad62f4d7283f7577e1d9b294a` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:41.701804 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70884f01-253e-4a57-9c24-aef4b9e52f72 None admin] RBAC: Authorizing `identity:get_user(user_id=1a1b079ad62f4d7283f7577e1d9b294a)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:41.703258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70884f01-253e-4a57-9c24-aef4b9e52f72 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:41.704829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 551/1127] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:41:41 2022] GET /identity/v3/users/1a1b079ad62f4d7283f7577e1d9b294a => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:41.710327 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cba86-7195-403c-b759-06bfe1889931 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:41.805436 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693cba86-7195-403c-b759-06bfe1889931 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:41.837464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0629f1c-597c-416e-99aa-e9b998f1e328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:41.837999 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0629f1c-597c-416e-99aa-e9b998f1e328 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:41.838276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0629f1c-597c-416e-99aa-e9b998f1e328 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:41.838570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0629f1c-597c-416e-99aa-e9b998f1e328 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:41.870272 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:41.903933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0629f1c-597c-416e-99aa-e9b998f1e328 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:41.905960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0629f1c-597c-416e-99aa-e9b998f1e328 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:41.999898 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 577/1128] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:41:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:41:42.005644 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70884f01-253e-4a57-9c24-aef4b9e52f72 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:42.101242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70884f01-253e-4a57-9c24-aef4b9e52f72 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:42.134072 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f1b05-101b-468c-87c5-8b6e8984c9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:42.134729 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f1b05-101b-468c-87c5-8b6e8984c9cc None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:42.135009 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f1b05-101b-468c-87c5-8b6e8984c9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:42.135282 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f1b05-101b-468c-87c5-8b6e8984c9cc None admin] PATH_INFO: `/v3/projects/90b79586a53c48d7b4519b349cf13299` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:42.136425 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f1b05-101b-468c-87c5-8b6e8984c9cc None admin] RBAC: Authorizing `identity:get_project(project_id=90b79586a53c48d7b4519b349cf13299)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:42.137839 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f1b05-101b-468c-87c5-8b6e8984c9cc None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:42.140115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 552/1129] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:41:42 2022] GET /identity/v3/projects/90b79586a53c48d7b4519b349cf13299 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:42.146245 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0629f1c-597c-416e-99aa-e9b998f1e328 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:42.209699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:42.242956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0629f1c-597c-416e-99aa-e9b998f1e328 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:42.276107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204ce3f-fc57-4c7b-bc2c-b28567d05fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:42.276709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204ce3f-fc57-4c7b-bc2c-b28567d05fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:42.276990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204ce3f-fc57-4c7b-bc2c-b28567d05fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:42.277265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204ce3f-fc57-4c7b-bc2c-b28567d05fe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:42.278043 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204ce3f-fc57-4c7b-bc2c-b28567d05fe2 None admin] RBAC: Adding query filter params (role.id=79ff845a41e241229433560e2b0d6fe5, scope.project.id=90b79586a53c48d7b4519b349cf13299, user.id=1a1b079ad62f4d7283f7577e1d9b294a) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:42.278354 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204ce3f-fc57-4c7b-bc2c-b28567d05fe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:42.279827 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204ce3f-fc57-4c7b-bc2c-b28567d05fe2 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:42.285425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 578/1130] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:41:42 2022] GET /identity/v3/role_assignments?user.id=1a1b079ad62f4d7283f7577e1d9b294a&scope.project.id=90b79586a53c48d7b4519b349cf13299&role.id=79ff845a41e241229433560e2b0d6fe5 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:41:44.264121 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe633aa0-ba9e-4c19-bb9a-e52bd191bd56 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:44.264671 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe633aa0-ba9e-4c19-bb9a-e52bd191bd56 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:44.264671 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe633aa0-ba9e-4c19-bb9a-e52bd191bd56 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:44.266025 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 553/1131] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:44.271220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d9a5-9e40-42cc-900a-287677878ac9 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:44.271605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d9a5-9e40-42cc-900a-287677878ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:44.271770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d9a5-9e40-42cc-900a-287677878ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:44.289632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-af60d9a5-9e40-42cc-900a-287677878ac9 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:44.290840 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-af60d9a5-9e40-42cc-900a-287677878ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:44.354124 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 579/1132] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:44.376621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe633aa0-ba9e-4c19-bb9a-e52bd191bd56 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:44.391670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:44.475291 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe633aa0-ba9e-4c19-bb9a-e52bd191bd56 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:44.507725 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ef554-b3fc-4cfc-8263-775e2564217c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:44.508345 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ef554-b3fc-4cfc-8263-775e2564217c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:44.508621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ef554-b3fc-4cfc-8263-775e2564217c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:44.508890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ef554-b3fc-4cfc-8263-775e2564217c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:44.510035 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ef554-b3fc-4cfc-8263-775e2564217c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:44.511478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ef554-b3fc-4cfc-8263-775e2564217c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:44.513040 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 554/1133] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:44.518787 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60d9a5-9e40-42cc-900a-287677878ac9 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:44.577646 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:44.616518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60d9a5-9e40-42cc-900a-287677878ac9 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:44.649004 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ad525-72a7-4bb9-9ac0-181231fbb3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:44.649603 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ad525-72a7-4bb9-9ac0-181231fbb3c1 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:44.649879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ad525-72a7-4bb9-9ac0-181231fbb3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:44.650146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ad525-72a7-4bb9-9ac0-181231fbb3c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:44.650851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ad525-72a7-4bb9-9ac0-181231fbb3c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:44.652253 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ad525-72a7-4bb9-9ac0-181231fbb3c1 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:44.678596 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 580/1134] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 3 12:41:44 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:41:46.667610 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bee2d-850d-48bc-81cc-ceaa0b67c3f4 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:46.668147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bee2d-850d-48bc-81cc-ceaa0b67c3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:46.668147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bee2d-850d-48bc-81cc-ceaa0b67c3f4 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:46.669527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 555/1135] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:46.674818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6426e24-1314-4400-b070-2f7597b769ca None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:46.675251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6426e24-1314-4400-b070-2f7597b769ca None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:46.675380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6426e24-1314-4400-b070-2f7597b769ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:46.693751 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-c6426e24-1314-4400-b070-2f7597b769ca None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:46.694963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c6426e24-1314-4400-b070-2f7597b769ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:46.758304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 581/1136] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:46.782755 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bee2d-850d-48bc-81cc-ceaa0b67c3f4 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:46.865719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:46.885162 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bee2d-850d-48bc-81cc-ceaa0b67c3f4 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:46.917402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cd003-d304-4b98-983e-628d80bfbe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:46.918080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cd003-d304-4b98-983e-628d80bfbe93 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:46.918315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cd003-d304-4b98-983e-628d80bfbe93 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:46.918622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cd003-d304-4b98-983e-628d80bfbe93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:46.919779 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cd003-d304-4b98-983e-628d80bfbe93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:46.921705 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cd003-d304-4b98-983e-628d80bfbe93 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:46.923316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 556/1137] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:41:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:46.929061 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6426e24-1314-4400-b070-2f7597b769ca None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:46.970827 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:47.025140 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6426e24-1314-4400-b070-2f7597b769ca None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:47.057589 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5697b-835a-4fc9-8372-0f21d622c743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:47.058194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5697b-835a-4fc9-8372-0f21d622c743 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:47.058755 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5697b-835a-4fc9-8372-0f21d622c743 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:47.059039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5697b-835a-4fc9-8372-0f21d622c743 None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:47.059720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5697b-835a-4fc9-8372-0f21d622c743 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:47.061128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5697b-835a-4fc9-8372-0f21d622c743 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:47.081350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 582/1138] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:41:46 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:41:49.098608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dfddb-4ebb-497f-bf46-e867eee0e024 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:49.099154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dfddb-4ebb-497f-bf46-e867eee0e024 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:49.099154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dfddb-4ebb-497f-bf46-e867eee0e024 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:49.100508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 557/1139] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:49.105883 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62154f97-9732-48f5-8033-cb1ff84e9dd7 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:49.106251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62154f97-9732-48f5-8033-cb1ff84e9dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:49.106438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62154f97-9732-48f5-8033-cb1ff84e9dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:49.124114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-62154f97-9732-48f5-8033-cb1ff84e9dd7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:49.125292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-62154f97-9732-48f5-8033-cb1ff84e9dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:49.189059 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 583/1140] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:49.211951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dfddb-4ebb-497f-bf46-e867eee0e024 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:49.309692 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dfddb-4ebb-497f-bf46-e867eee0e024 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:49.342207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cf37e-4eb3-431c-8bf7-a15656427b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:49.343398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cf37e-4eb3-431c-8bf7-a15656427b27 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:49.343675 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cf37e-4eb3-431c-8bf7-a15656427b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:49.343945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cf37e-4eb3-431c-8bf7-a15656427b27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:49.353734 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265cf37e-4eb3-431c-8bf7-a15656427b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:41:49.358581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-265cf37e-4eb3-431c-8bf7-a15656427b27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:41:49.358581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:49.358581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:49.358581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:49.358581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:49.358581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:49.358581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:49.358581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:49.358581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:49.358581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:49.358581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:49.358581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:49.358581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:41:49.358581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:41:49.358581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:41:49.358581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:41:49.359759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:41:49.359759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:41:49.359759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:41:49.359759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:41:49.359759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:49.359759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:49.359759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:49.359759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:49.359759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:49.359759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:49.359759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:49.359759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:49.359759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:49.359759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:49.359759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:49.359759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:49.360853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:49.360853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:49.360853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:49.360853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:49.360853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:41:49.360853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:41:49.360853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:41:49.360853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:41:49.360853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:41:49.360853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:41:49.360853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:41:49.360853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:49.360853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 558/1141] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:41:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:49.365392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62154f97-9732-48f5-8033-cb1ff84e9dd7 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:49.392363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:49.463231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62154f97-9732-48f5-8033-cb1ff84e9dd7 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:49.495276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fe4e5-5e16-4506-9191-c2bc8c1b5f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:49.495876 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fe4e5-5e16-4506-9191-c2bc8c1b5f8f None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:49.496159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fe4e5-5e16-4506-9191-c2bc8c1b5f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:49.496434 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fe4e5-5e16-4506-9191-c2bc8c1b5f8f None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:49.497144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fe4e5-5e16-4506-9191-c2bc8c1b5f8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:49.497450 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fe4e5-5e16-4506-9191-c2bc8c1b5f8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:49.499349 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fe4e5-5e16-4506-9191-c2bc8c1b5f8f None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:49.507073 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 584/1142] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:41:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:49.512513 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cf37e-4eb3-431c-8bf7-a15656427b27 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:49.525386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:49.609441 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cf37e-4eb3-431c-8bf7-a15656427b27 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:49.643069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174a42b-f7dd-40c3-9416-d65bb4761440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:49.643685 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174a42b-f7dd-40c3-9416-d65bb4761440 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:49.643962 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174a42b-f7dd-40c3-9416-d65bb4761440 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:49.644230 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174a42b-f7dd-40c3-9416-d65bb4761440 None admin] PATH_INFO: `/v3/users/d61a0f3f5de84c74a5b33a4a4d373948` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:49.656700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174a42b-f7dd-40c3-9416-d65bb4761440 None admin] RBAC: Authorizing `identity:get_user(user_id=d61a0f3f5de84c74a5b33a4a4d373948)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:49.658114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174a42b-f7dd-40c3-9416-d65bb4761440 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:49.659725 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 559/1143] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:41:49 2022] GET /identity/v3/users/d61a0f3f5de84c74a5b33a4a4d373948 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:49.665395 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fe4e5-5e16-4506-9191-c2bc8c1b5f8f None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:49.753169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:49.762801 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fe4e5-5e16-4506-9191-c2bc8c1b5f8f None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:49.794704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff3c89-0c85-4a04-9d5c-b1ab2ecfba1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:49.795302 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff3c89-0c85-4a04-9d5c-b1ab2ecfba1e None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:49.795580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff3c89-0c85-4a04-9d5c-b1ab2ecfba1e None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:49.796335 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff3c89-0c85-4a04-9d5c-b1ab2ecfba1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:49.860342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff3c89-0c85-4a04-9d5c-b1ab2ecfba1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:49.861762 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff3c89-0c85-4a04-9d5c-b1ab2ecfba1e None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:49.945450 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:49.955701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 585/1144] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:41:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:41:49.961504 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174a42b-f7dd-40c3-9416-d65bb4761440 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:50.024914 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:50.056539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174a42b-f7dd-40c3-9416-d65bb4761440 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:50.089251 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a578ba4-3403-40e8-b797-e8c24626c998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:50.089875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a578ba4-3403-40e8-b797-e8c24626c998 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:50.090155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a578ba4-3403-40e8-b797-e8c24626c998 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:50.090426 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a578ba4-3403-40e8-b797-e8c24626c998 None admin] PATH_INFO: `/v3/projects/f46576fd813b4813b346842b723787f2` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:50.091604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a578ba4-3403-40e8-b797-e8c24626c998 None admin] RBAC: Authorizing `identity:get_project(project_id=f46576fd813b4813b346842b723787f2)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:50.093058 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a578ba4-3403-40e8-b797-e8c24626c998 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:50.094721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 560/1145] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:41:49 2022] GET /identity/v3/projects/f46576fd813b4813b346842b723787f2 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:50.100658 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff3c89-0c85-4a04-9d5c-b1ab2ecfba1e None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:50.195654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff3c89-0c85-4a04-9d5c-b1ab2ecfba1e None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:50.229516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b85da-d5a9-4c03-96fc-4a5ed0916cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:50.230115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b85da-d5a9-4c03-96fc-4a5ed0916cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:50.230395 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b85da-d5a9-4c03-96fc-4a5ed0916cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:50.230691 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b85da-d5a9-4c03-96fc-4a5ed0916cb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:50.231467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b85da-d5a9-4c03-96fc-4a5ed0916cb1 None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=f46576fd813b4813b346842b723787f2, user.id=d61a0f3f5de84c74a5b33a4a4d373948) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:50.231775 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b85da-d5a9-4c03-96fc-4a5ed0916cb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:50.233772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b85da-d5a9-4c03-96fc-4a5ed0916cb1 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:50.239036 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 586/1146] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:41:50 2022] GET /identity/v3/role_assignments?user.id=d61a0f3f5de84c74a5b33a4a4d373948&scope.project.id=f46576fd813b4813b346842b723787f2&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:52.172267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fe1ab-3103-4256-8485-fbef24896fb7 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:52.172783 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fe1ab-3103-4256-8485-fbef24896fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:52.172783 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8fe1ab-3103-4256-8485-fbef24896fb7 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:52.174071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 561/1147] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:41:52.179292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b38bc-73ff-4b3d-ad9f-8da355749fb2 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:52.179673 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b38bc-73ff-4b3d-ad9f-8da355749fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:52.179835 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b38bc-73ff-4b3d-ad9f-8da355749fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:52.197490 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-a37b38bc-73ff-4b3d-ad9f-8da355749fb2 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:52.198692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a37b38bc-73ff-4b3d-ad9f-8da355749fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:52.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:52.264945 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 587/1148] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:52.286912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fe1ab-3103-4256-8485-fbef24896fb7 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:52.383908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8fe1ab-3103-4256-8485-fbef24896fb7 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:52.416812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34d839-5684-4ffd-a3e8-ccaa14c28acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:52.417430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34d839-5684-4ffd-a3e8-ccaa14c28acd None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:52.417710 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34d839-5684-4ffd-a3e8-ccaa14c28acd None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:52.417984 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34d839-5684-4ffd-a3e8-ccaa14c28acd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:52.420541 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:52.428453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34d839-5684-4ffd-a3e8-ccaa14c28acd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:41:52.433257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-dd34d839-5684-4ffd-a3e8-ccaa14c28acd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:41:52.433257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:52.433257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:52.433257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:52.433257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:52.433257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:52.433257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:52.433257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:52.433257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:52.433257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:52.433257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:52.433257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:52.433257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:41:52.433257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:41:52.433257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:41:52.433257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:41:52.434719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:41:52.434719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:41:52.434719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:41:52.434719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:41:52.434719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:52.434719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:52.434719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:52.434719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:52.434719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:52.434719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:52.434719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:52.434719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:52.434719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:52.434719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:52.434719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:52.434719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:52.435777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:52.435777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:52.435777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:52.435777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:52.435777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:41:52.435777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:41:52.435777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:41:52.435777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:41:52.435777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:41:52.435777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:41:52.435777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:41:52.435777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:52.435777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 562/1149] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:41:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:52.440967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b38bc-73ff-4b3d-ad9f-8da355749fb2 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:52.536548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b38bc-73ff-4b3d-ad9f-8da355749fb2 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:52.568945 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432464b-628a-40fe-8b81-3b51699c9bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:52.569547 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432464b-628a-40fe-8b81-3b51699c9bfe None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:52.569826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432464b-628a-40fe-8b81-3b51699c9bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:52.570098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432464b-628a-40fe-8b81-3b51699c9bfe None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:52.570838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432464b-628a-40fe-8b81-3b51699c9bfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:52.571139 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432464b-628a-40fe-8b81-3b51699c9bfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:52.572488 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432464b-628a-40fe-8b81-3b51699c9bfe None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:52.580799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 588/1150] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:41:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:52.586296 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34d839-5684-4ffd-a3e8-ccaa14c28acd None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:52.680305 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34d839-5684-4ffd-a3e8-ccaa14c28acd None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:52.713215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a3d0d-a6a1-439f-8a53-54f05b8922dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:52.713824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a3d0d-a6a1-439f-8a53-54f05b8922dc None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:52.714100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a3d0d-a6a1-439f-8a53-54f05b8922dc None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:52.714368 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a3d0d-a6a1-439f-8a53-54f05b8922dc None admin] PATH_INFO: `/v3/users/d61a0f3f5de84c74a5b33a4a4d373948` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:52.715541 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a3d0d-a6a1-439f-8a53-54f05b8922dc None admin] RBAC: Authorizing `identity:get_user(user_id=d61a0f3f5de84c74a5b33a4a4d373948)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:52.716945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a3d0d-a6a1-439f-8a53-54f05b8922dc None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:52.718538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 563/1151] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:41:52 2022] GET /identity/v3/users/d61a0f3f5de84c74a5b33a4a4d373948 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:52.724067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432464b-628a-40fe-8b81-3b51699c9bfe None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:52.769199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:52.820734 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432464b-628a-40fe-8b81-3b51699c9bfe None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:52.852327 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1d402-f85b-48f2-a5c3-d892fa0ad448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:52.852955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1d402-f85b-48f2-a5c3-d892fa0ad448 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:52.853234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1d402-f85b-48f2-a5c3-d892fa0ad448 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:52.853510 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1d402-f85b-48f2-a5c3-d892fa0ad448 None admin] PATH_INFO: `/v3/projects/f46576fd813b4813b346842b723787f2` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:52.854682 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1d402-f85b-48f2-a5c3-d892fa0ad448 None admin] RBAC: Authorizing `identity:get_project(project_id=f46576fd813b4813b346842b723787f2)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:52.856122 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda1d402-f85b-48f2-a5c3-d892fa0ad448 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:52.857769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 589/1152] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:41:52 2022] GET /identity/v3/projects/f46576fd813b4813b346842b723787f2 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:52.864333 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a3d0d-a6a1-439f-8a53-54f05b8922dc None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:52.924695 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:52.961596 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a3d0d-a6a1-439f-8a53-54f05b8922dc None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:52.994312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f6c9b-f8db-4d40-a176-edc0f88758ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:52.994992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6c9b-f8db-4d40-a176-edc0f88758ba None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:52.995276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6c9b-f8db-4d40-a176-edc0f88758ba None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:52.995552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6c9b-f8db-4d40-a176-edc0f88758ba None admin] PATH_INFO: `/v3/projects/f46576fd813b4813b346842b723787f2/users/d61a0f3f5de84c74a5b33a4a4d373948/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:52.997482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f6c9b-f8db-4d40-a176-edc0f88758ba None admin] RBAC: Authorizing `identity:create_grant(project_id=f46576fd813b4813b346842b723787f2, user_id=d61a0f3f5de84c74a5b33a4a4d373948, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:52.999530 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f6c9b-f8db-4d40-a176-edc0f88758ba None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:53.011996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 564/1153] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:41:52 2022] PUT /identity/v3/projects/f46576fd813b4813b346842b723787f2/users/d61a0f3f5de84c74a5b33a4a4d373948/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:41:54.960169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed51e7e-fac2-4393-bfe9-85f21a872dae None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:54.960650 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed51e7e-fac2-4393-bfe9-85f21a872dae None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:54.960755 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed51e7e-fac2-4393-bfe9-85f21a872dae None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:54.962042 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 590/1154] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:54.967146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6e482-7613-4a07-8745-957a80e0446d None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:54.968023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6e482-7613-4a07-8745-957a80e0446d None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:54.968218 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6e482-7613-4a07-8745-957a80e0446d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:54.986346 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-fec6e482-7613-4a07-8745-957a80e0446d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:54.987548 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-fec6e482-7613-4a07-8745-957a80e0446d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:55.051428 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 565/1155] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:55.074045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed51e7e-fac2-4393-bfe9-85f21a872dae None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:55.173235 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed51e7e-fac2-4393-bfe9-85f21a872dae None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:55.186848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:55.206626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ca958-4b73-4ed9-bb82-e996bf0b6e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:55.207252 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ca958-4b73-4ed9-bb82-e996bf0b6e8b None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:55.207529 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ca958-4b73-4ed9-bb82-e996bf0b6e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:55.207799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ca958-4b73-4ed9-bb82-e996bf0b6e8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:55.218064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ca958-4b73-4ed9-bb82-e996bf0b6e8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:41:55.222943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-1c0ca958-4b73-4ed9-bb82-e996bf0b6e8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:41:55.222943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:55.222943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:55.222943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:55.222943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:55.222943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:55.222943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:55.222943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:55.222943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:55.222943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:55.222943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:55.222943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:55.222943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:41:55.222943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:41:55.222943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:41:55.222943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:41:55.224055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:41:55.224055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:41:55.224055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:41:55.224055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:41:55.224055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:55.224055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:55.224055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:55.224055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:55.224055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:55.224055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:55.224055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:55.224055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:55.224055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:55.224055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:55.224055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:55.224055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:55.225129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:55.225129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:55.225129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:55.225129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:55.225129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:41:55.225129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:41:55.225129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:41:55.225129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:41:55.225129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:41:55.225129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:41:55.225129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:41:55.225129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:41:55.225129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 591/1156] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:41:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:55.229939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6e482-7613-4a07-8745-957a80e0446d None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:55.282903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:55.325617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec6e482-7613-4a07-8745-957a80e0446d None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:55.358580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15825961-8d86-4870-8226-e3e747a76fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:55.359178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15825961-8d86-4870-8226-e3e747a76fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:55.359458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15825961-8d86-4870-8226-e3e747a76fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:55.359731 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15825961-8d86-4870-8226-e3e747a76fc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:55.360437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15825961-8d86-4870-8226-e3e747a76fc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:55.360738 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15825961-8d86-4870-8226-e3e747a76fc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:55.362097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15825961-8d86-4870-8226-e3e747a76fc8 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:55.369811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 566/1157] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:41:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:55.375364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ca958-4b73-4ed9-bb82-e996bf0b6e8b None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:55.471230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ca958-4b73-4ed9-bb82-e996bf0b6e8b None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:55.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1b59f-9f62-497d-a49a-0501e1a100e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:55.504197 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1b59f-9f62-497d-a49a-0501e1a100e3 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:55.504478 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1b59f-9f62-497d-a49a-0501e1a100e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:55.504752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1b59f-9f62-497d-a49a-0501e1a100e3 None admin] PATH_INFO: `/v3/users/d61a0f3f5de84c74a5b33a4a4d373948` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:55.505903 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1b59f-9f62-497d-a49a-0501e1a100e3 None admin] RBAC: Authorizing `identity:get_user(user_id=d61a0f3f5de84c74a5b33a4a4d373948)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:55.507340 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1b59f-9f62-497d-a49a-0501e1a100e3 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:55.508892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 592/1158] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:41:55 2022] GET /identity/v3/users/d61a0f3f5de84c74a5b33a4a4d373948 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:55.514344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15825961-8d86-4870-8226-e3e747a76fc8 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:55.612493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15825961-8d86-4870-8226-e3e747a76fc8 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:55.628137 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:55.645962 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e4759-2959-4f6a-ad66-4eb8bb37d21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:55.646593 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e4759-2959-4f6a-ad66-4eb8bb37d21f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:55.646874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e4759-2959-4f6a-ad66-4eb8bb37d21f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:55.647146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e4759-2959-4f6a-ad66-4eb8bb37d21f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:55.711462 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e4759-2959-4f6a-ad66-4eb8bb37d21f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:55.712891 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e4759-2959-4f6a-ad66-4eb8bb37d21f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:55.806774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 567/1159] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:41:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:41:55.813550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1b59f-9f62-497d-a49a-0501e1a100e3 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:55.855619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:55.912671 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1b59f-9f62-497d-a49a-0501e1a100e3 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:55.945478 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9275bb-5e6c-46f0-b458-9c82d20007d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:55.946102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9275bb-5e6c-46f0-b458-9c82d20007d7 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:55.946378 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9275bb-5e6c-46f0-b458-9c82d20007d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:55.946679 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9275bb-5e6c-46f0-b458-9c82d20007d7 None admin] PATH_INFO: `/v3/projects/f46576fd813b4813b346842b723787f2` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:55.947916 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9275bb-5e6c-46f0-b458-9c82d20007d7 None admin] RBAC: Authorizing `identity:get_project(project_id=f46576fd813b4813b346842b723787f2)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:55.949350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9275bb-5e6c-46f0-b458-9c82d20007d7 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:55.951056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 593/1160] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:41:55 2022] GET /identity/v3/projects/f46576fd813b4813b346842b723787f2 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:55.957177 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e4759-2959-4f6a-ad66-4eb8bb37d21f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:55.973971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:56.052873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e4759-2959-4f6a-ad66-4eb8bb37d21f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:56.086057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1374048-b184-4c90-a293-4eeaecffe2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:56.086638 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1374048-b184-4c90-a293-4eeaecffe2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:56.086921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1374048-b184-4c90-a293-4eeaecffe2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:56.087191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1374048-b184-4c90-a293-4eeaecffe2f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:56.087973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1374048-b184-4c90-a293-4eeaecffe2f1 None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=f46576fd813b4813b346842b723787f2, user.id=d61a0f3f5de84c74a5b33a4a4d373948) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:56.088280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1374048-b184-4c90-a293-4eeaecffe2f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:56.089687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1374048-b184-4c90-a293-4eeaecffe2f1 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:56.095433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 568/1161] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:41:55 2022] GET /identity/v3/role_assignments?user.id=d61a0f3f5de84c74a5b33a4a4d373948&scope.project.id=f46576fd813b4813b346842b723787f2&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:41:58.056493 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4450f7-639e-45e5-9ee1-39afa99490d7 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:58.057010 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4450f7-639e-45e5-9ee1-39afa99490d7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:58.057096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4450f7-639e-45e5-9ee1-39afa99490d7 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:58.058670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 594/1162] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:41:58.063789 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a6dcb-85d3-4d12-b7b2-57f6fd47a46c None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:58.064159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a6dcb-85d3-4d12-b7b2-57f6fd47a46c None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:58.064333 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a6dcb-85d3-4d12-b7b2-57f6fd47a46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:58.082284 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a10a6dcb-85d3-4d12-b7b2-57f6fd47a46c None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:41:58.083493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a10a6dcb-85d3-4d12-b7b2-57f6fd47a46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:41:58.146366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 569/1163] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:41:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:41:58.168141 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4450f7-639e-45e5-9ee1-39afa99490d7 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:58.265701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4450f7-639e-45e5-9ee1-39afa99490d7 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:58.275793 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:58.299150 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408607f-4eb6-4ee4-8e8e-0004f07eabf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:58.299754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408607f-4eb6-4ee4-8e8e-0004f07eabf8 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:58.300034 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408607f-4eb6-4ee4-8e8e-0004f07eabf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:58.300306 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b408607f-4eb6-4ee4-8e8e-0004f07eabf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:58.364719 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408607f-4eb6-4ee4-8e8e-0004f07eabf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:58.366137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b408607f-4eb6-4ee4-8e8e-0004f07eabf8 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:58.460140 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 595/1164] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:41:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:41:58.466153 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a6dcb-85d3-4d12-b7b2-57f6fd47a46c None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:58.477266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:58.562221 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a6dcb-85d3-4d12-b7b2-57f6fd47a46c None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:58.594758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff05779c-768a-4d51-9754-4fa4f01424c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:58.595382 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05779c-768a-4d51-9754-4fa4f01424c4 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:58.595661 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05779c-768a-4d51-9754-4fa4f01424c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:58.595930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff05779c-768a-4d51-9754-4fa4f01424c4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:58.602298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05779c-768a-4d51-9754-4fa4f01424c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:58.603753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff05779c-768a-4d51-9754-4fa4f01424c4 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-ff05779c-768a-4d51-9754-4fa4f01424c4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:41:58.615101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:41:58.616344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:41:58.616344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:41:58.616344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:41:58.616344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:41:58.616344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:41:58.616344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:41:58.616344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:41:58.616344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:41:58.616344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:41:58.616344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:41:58.616344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:41:58.616344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:41:58.616344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:41:58.616344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:41:58.616344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:41:58.616344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:41:58.617262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:41:58.617262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:41:58.617262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:41:58.617262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:41:58.617262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:41:58.617262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:41:58.617262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:41:58.617262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:41:58.617262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:41:58.617262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:41:58.617262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 03 12:41:58.617262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:41:58.617262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 03 12:41:58.617262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:41:58.617262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 570/1165] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 3 12:41:58 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:41:58.622042 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408607f-4eb6-4ee4-8e8e-0004f07eabf8 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:41:58.633584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:41:58.716998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b408607f-4eb6-4ee4-8e8e-0004f07eabf8 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:41:58.748345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bfa80-7fce-4498-a754-7582254f47d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:41:58.749455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bfa80-7fce-4498-a754-7582254f47d8 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:41:58.749732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bfa80-7fce-4498-a754-7582254f47d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:41:58.750000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bfa80-7fce-4498-a754-7582254f47d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:41:58.750700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bfa80-7fce-4498-a754-7582254f47d8 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:41:58.751004 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bfa80-7fce-4498-a754-7582254f47d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:41:58.752334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bfa80-7fce-4498-a754-7582254f47d8 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:41:58.759071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 596/1166] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 3 12:41:58 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:42:00.715825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f880d29-e385-42d6-bc73-87da276009df None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:00.716360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f880d29-e385-42d6-bc73-87da276009df None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:00.716360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f880d29-e385-42d6-bc73-87da276009df None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:00.717704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 571/1167] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:42:00.722740 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdfd07-3140-4863-bf46-3757ee49c85a None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:00.723176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdfd07-3140-4863-bf46-3757ee49c85a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:00.723279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdfd07-3140-4863-bf46-3757ee49c85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:00.740974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-92bdfd07-3140-4863-bf46-3757ee49c85a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:42:00.742168 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-92bdfd07-3140-4863-bf46-3757ee49c85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:42:00.806335 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 597/1168] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:42:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:42:00.829686 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f880d29-e385-42d6-bc73-87da276009df None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:00.897871 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:00.928182 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f880d29-e385-42d6-bc73-87da276009df None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:00.961025 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef3d8ea-acc7-4241-9def-db7f3a8c0cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:00.961627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3d8ea-acc7-4241-9def-db7f3a8c0cc7 None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:00.961903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3d8ea-acc7-4241-9def-db7f3a8c0cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:00.962173 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef3d8ea-acc7-4241-9def-db7f3a8c0cc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:00.962912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3d8ea-acc7-4241-9def-db7f3a8c0cc7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:00.964257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef3d8ea-acc7-4241-9def-db7f3a8c0cc7 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:00.979180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 572/1169] 10.222.0.65 () {60 vars in 1138 bytes} [Wed Aug 3 12:42:00 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:42:02.969087 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4d601-d930-44b2-aa4d-2cdc08292034 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:02.969634 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4d601-d930-44b2-aa4d-2cdc08292034 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:02.969634 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4d601-d930-44b2-aa4d-2cdc08292034 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:02.971013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 598/1170] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:42:02.976145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70829c4d-2eb8-4d23-ad74-22195c3ac830 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:02.976558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70829c4d-2eb8-4d23-ad74-22195c3ac830 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:02.976674 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70829c4d-2eb8-4d23-ad74-22195c3ac830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:02.994198 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-70829c4d-2eb8-4d23-ad74-22195c3ac830 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:42:02.995391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-70829c4d-2eb8-4d23-ad74-22195c3ac830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:42:03.059353 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 573/1171] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:42:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:42:03.083044 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4d601-d930-44b2-aa4d-2cdc08292034 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:03.086737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:03.179296 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4d601-d930-44b2-aa4d-2cdc08292034 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:03.211999 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e84811-e166-4240-8883-c04590d6aee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:03.212617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e84811-e166-4240-8883-c04590d6aee5 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:03.212893 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e84811-e166-4240-8883-c04590d6aee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:03.213161 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e84811-e166-4240-8883-c04590d6aee5 None admin] PATH_INFO: `/v3/domains/318cafacd06a43598357b9eae355b878` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:03.223030 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e84811-e166-4240-8883-c04590d6aee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=318cafacd06a43598357b9eae355b878)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:03.224448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e84811-e166-4240-8883-c04590d6aee5 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:03.226001 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 599/1172] 10.222.0.65 () {56 vars in 1183 bytes} [Wed Aug 3 12:42:03 2022] GET /identity/v3/domains/318cafacd06a43598357b9eae355b878 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:42:03.231747 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70829c4d-2eb8-4d23-ad74-22195c3ac830 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:03.287959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:03.328583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70829c4d-2eb8-4d23-ad74-22195c3ac830 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:03.361242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb10944-8d3c-4319-b25c-af7541bb86af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:03.361846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb10944-8d3c-4319-b25c-af7541bb86af None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:03.362122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb10944-8d3c-4319-b25c-af7541bb86af None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:03.362390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb10944-8d3c-4319-b25c-af7541bb86af None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:03.363099 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb10944-8d3c-4319-b25c-af7541bb86af None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:03.364499 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb10944-8d3c-4319-b25c-af7541bb86af None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:03.389518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 574/1173] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 3 12:42:03 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:42:05.362972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02159992-450d-4afc-98b6-5b4ceabcd18f None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:05.363520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02159992-450d-4afc-98b6-5b4ceabcd18f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:05.363520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02159992-450d-4afc-98b6-5b4ceabcd18f None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:05.364830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 600/1174] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:42:05.369953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef23654-01f7-401b-952f-f38ae6e773c5 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:05.370329 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef23654-01f7-401b-952f-f38ae6e773c5 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:05.370519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef23654-01f7-401b-952f-f38ae6e773c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:05.388262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-cef23654-01f7-401b-952f-f38ae6e773c5 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:42:05.389461 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-cef23654-01f7-401b-952f-f38ae6e773c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:42:05.452618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 575/1175] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:42:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:42:05.476785 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02159992-450d-4afc-98b6-5b4ceabcd18f None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:05.537149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:05.575119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02159992-450d-4afc-98b6-5b4ceabcd18f None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:05.607511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df68677-0242-4876-976f-936ee1ff96b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:05.608151 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df68677-0242-4876-976f-936ee1ff96b5 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:05.608427 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df68677-0242-4876-976f-936ee1ff96b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:05.608697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df68677-0242-4876-976f-936ee1ff96b5 None admin] PATH_INFO: `/v3/domains/318cafacd06a43598357b9eae355b878` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:05.609821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df68677-0242-4876-976f-936ee1ff96b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=318cafacd06a43598357b9eae355b878)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:05.611251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df68677-0242-4876-976f-936ee1ff96b5 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:05.612828 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 601/1176] 10.222.0.65 () {56 vars in 1183 bytes} [Wed Aug 3 12:42:05 2022] GET /identity/v3/domains/318cafacd06a43598357b9eae355b878 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:42:05.618710 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef23654-01f7-401b-952f-f38ae6e773c5 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:05.659837 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:05.714707 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef23654-01f7-401b-952f-f38ae6e773c5 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:05.747877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d966551-a62d-4fc2-b3ea-3473e65eb427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:05.748483 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d966551-a62d-4fc2-b3ea-3473e65eb427 None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:05.748765 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d966551-a62d-4fc2-b3ea-3473e65eb427 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:05.749041 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d966551-a62d-4fc2-b3ea-3473e65eb427 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:05.749735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d966551-a62d-4fc2-b3ea-3473e65eb427 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:05.751191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d966551-a62d-4fc2-b3ea-3473e65eb427 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:05.772529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 576/1177] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:42:05 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:42:07.781513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a19f0-30a4-4f20-a20c-a19814d07b29 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:07.783442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a19f0-30a4-4f20-a20c-a19814d07b29 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:07.783442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a19f0-30a4-4f20-a20c-a19814d07b29 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:07.783442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 602/1178] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:42:07.788508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfabe5-b1fc-4a79-8a3a-c447a41d3765 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:07.788909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfabe5-b1fc-4a79-8a3a-c447a41d3765 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:07.789059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfabe5-b1fc-4a79-8a3a-c447a41d3765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:07.807465 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-8bcfabe5-b1fc-4a79-8a3a-c447a41d3765 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:42:07.808895 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8bcfabe5-b1fc-4a79-8a3a-c447a41d3765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:42:07.872374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 577/1179] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:42:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:42:07.895347 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a19f0-30a4-4f20-a20c-a19814d07b29 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:07.991114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a19f0-30a4-4f20-a20c-a19814d07b29 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:08.019000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:08.024266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9d87e-9869-48c2-8dc9-ca4dafe12875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:08.024887 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9d87e-9869-48c2-8dc9-ca4dafe12875 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:08.025163 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9d87e-9869-48c2-8dc9-ca4dafe12875 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:08.025495 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9d87e-9869-48c2-8dc9-ca4dafe12875 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:08.035266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9d87e-9869-48c2-8dc9-ca4dafe12875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:42:08.040008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-fdf9d87e-9869-48c2-8dc9-ca4dafe12875 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:42:08.040008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:42:08.040008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:42:08.040008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:42:08.040008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:42:08.040008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:42:08.040008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:42:08.040008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:42:08.040008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:42:08.040008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:42:08.040008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:42:08.040008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:42:08.040008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:42:08.040008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:42:08.040008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:42:08.040008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:42:08.041070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:42:08.041070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:42:08.041070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:42:08.041070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:42:08.041070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:42:08.041070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:42:08.041070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:42:08.041070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:42:08.041070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:42:08.041070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:42:08.041070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:42:08.041070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:42:08.041070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:42:08.041070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:42:08.041070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:42:08.041070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:42:08.042381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:42:08.042381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:42:08.042381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:42:08.042381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:42:08.042381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:42:08.042381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:42:08.042381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:42:08.042381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:42:08.042381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:42:08.042381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:42:08.042381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:42:08.042381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:42:08.042381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 603/1180] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:42:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:42:08.047481 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcfabe5-b1fc-4a79-8a3a-c447a41d3765 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:08.074561 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:08.143364 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcfabe5-b1fc-4a79-8a3a-c447a41d3765 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:08.175710 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e41743-d2ac-49f2-961b-ff4b6603d0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:08.176315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e41743-d2ac-49f2-961b-ff4b6603d0eb None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:08.176595 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e41743-d2ac-49f2-961b-ff4b6603d0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:08.176865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e41743-d2ac-49f2-961b-ff4b6603d0eb None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:08.177574 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e41743-d2ac-49f2-961b-ff4b6603d0eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:42:08.177876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e41743-d2ac-49f2-961b-ff4b6603d0eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:08.179260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e41743-d2ac-49f2-961b-ff4b6603d0eb None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:08.187863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 578/1181] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:42:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:42:08.193617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9d87e-9869-48c2-8dc9-ca4dafe12875 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:08.292062 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9d87e-9869-48c2-8dc9-ca4dafe12875 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:08.324839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd252acf-bb1d-46ec-b41e-eee6cfe1d608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:08.325461 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd252acf-bb1d-46ec-b41e-eee6cfe1d608 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:08.325742 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd252acf-bb1d-46ec-b41e-eee6cfe1d608 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:08.326016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd252acf-bb1d-46ec-b41e-eee6cfe1d608 None admin] PATH_INFO: `/v3/users/67dae0a48b5443ad86952fc38224d990` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:08.338583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd252acf-bb1d-46ec-b41e-eee6cfe1d608 None admin] RBAC: Authorizing `identity:get_user(user_id=67dae0a48b5443ad86952fc38224d990)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:08.340028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd252acf-bb1d-46ec-b41e-eee6cfe1d608 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:08.341626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 604/1182] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:42:08 2022] GET /identity/v3/users/67dae0a48b5443ad86952fc38224d990 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:42:08.347838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e41743-d2ac-49f2-961b-ff4b6603d0eb None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:08.436992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:08.446609 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e41743-d2ac-49f2-961b-ff4b6603d0eb None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:08.478725 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80921b31-4246-48c9-aaf6-c717a934a492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:08.479327 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80921b31-4246-48c9-aaf6-c717a934a492 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:08.479606 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80921b31-4246-48c9-aaf6-c717a934a492 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:08.479877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80921b31-4246-48c9-aaf6-c717a934a492 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:08.543774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80921b31-4246-48c9-aaf6-c717a934a492 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:08.545708 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80921b31-4246-48c9-aaf6-c717a934a492 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:08.630954 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:08.641715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 579/1183] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:42:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:42:08.647488 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd252acf-bb1d-46ec-b41e-eee6cfe1d608 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:08.695638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:08.743800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd252acf-bb1d-46ec-b41e-eee6cfe1d608 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:08.776424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b637b-3c22-4cc4-ae6b-ea6f2768e73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:08.777057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b637b-3c22-4cc4-ae6b-ea6f2768e73f None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:08.777337 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b637b-3c22-4cc4-ae6b-ea6f2768e73f None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:08.777615 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b637b-3c22-4cc4-ae6b-ea6f2768e73f None admin] PATH_INFO: `/v3/projects/67440e36640a4835a6f2f77e89a3ee63` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:08.778788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b637b-3c22-4cc4-ae6b-ea6f2768e73f None admin] RBAC: Authorizing `identity:get_project(project_id=67440e36640a4835a6f2f77e89a3ee63)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:08.780816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b637b-3c22-4cc4-ae6b-ea6f2768e73f None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:08.782443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 605/1184] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:42:08 2022] GET /identity/v3/projects/67440e36640a4835a6f2f77e89a3ee63 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:42:08.788376 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80921b31-4246-48c9-aaf6-c717a934a492 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:08.883012 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80921b31-4246-48c9-aaf6-c717a934a492 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:08.916287 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0109d63-c789-40f7-aa9e-74111c50ee0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:08.916895 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0109d63-c789-40f7-aa9e-74111c50ee0e None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:08.917173 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0109d63-c789-40f7-aa9e-74111c50ee0e None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:08.917446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0109d63-c789-40f7-aa9e-74111c50ee0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:08.918225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0109d63-c789-40f7-aa9e-74111c50ee0e None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=67440e36640a4835a6f2f77e89a3ee63, user.id=67dae0a48b5443ad86952fc38224d990) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:42:08.918565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0109d63-c789-40f7-aa9e-74111c50ee0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:08.919985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0109d63-c789-40f7-aa9e-74111c50ee0e None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:08.925232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 580/1185] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:42:08 2022] GET /identity/v3/role_assignments?user.id=67dae0a48b5443ad86952fc38224d990&scope.project.id=67440e36640a4835a6f2f77e89a3ee63&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:42:10.864273 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0af30a-e335-4fa3-b2ed-8dd0dbe9f69b None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:10.864817 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0af30a-e335-4fa3-b2ed-8dd0dbe9f69b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:10.864817 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0af30a-e335-4fa3-b2ed-8dd0dbe9f69b None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:10.866137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 606/1186] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:42:10.871191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17743479-a778-4f02-bce6-c2594f1247f9 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:10.871609 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17743479-a778-4f02-bce6-c2594f1247f9 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:10.871753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17743479-a778-4f02-bce6-c2594f1247f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:10.890654 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-17743479-a778-4f02-bce6-c2594f1247f9 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:42:10.891835 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-17743479-a778-4f02-bce6-c2594f1247f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:42:10.939593 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:10.956765 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 581/1187] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:42:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:42:10.978439 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0af30a-e335-4fa3-b2ed-8dd0dbe9f69b None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:11.075625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0af30a-e335-4fa3-b2ed-8dd0dbe9f69b None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:11.091681 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:11.108715 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d81b1-0cbb-4e5f-b5d7-c1a91b7c021a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:11.109332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d81b1-0cbb-4e5f-b5d7-c1a91b7c021a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:11.109606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d81b1-0cbb-4e5f-b5d7-c1a91b7c021a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:11.109870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d81b1-0cbb-4e5f-b5d7-c1a91b7c021a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:11.119525 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d81b1-0cbb-4e5f-b5d7-c1a91b7c021a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:42:11.124882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-c18d81b1-0cbb-4e5f-b5d7-c1a91b7c021a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:42:11.124882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:42:11.124882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:42:11.124882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:42:11.124882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:42:11.124882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:42:11.124882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:42:11.124882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:42:11.124882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:42:11.124882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:42:11.124882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:42:11.124882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:42:11.124882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:42:11.124882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:42:11.124882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:42:11.124882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:42:11.125992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:42:11.125992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:42:11.125992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:42:11.125992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:42:11.125992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:42:11.125992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:42:11.125992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:42:11.125992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:42:11.125992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:42:11.125992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:42:11.125992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:42:11.125992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:42:11.125992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:42:11.125992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:42:11.125992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:42:11.125992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:42:11.127083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:42:11.127083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:42:11.127083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:42:11.127083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:42:11.127083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:42:11.127083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:42:11.127083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:42:11.127083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:42:11.127083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:42:11.127083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:42:11.127083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:42:11.127083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:42:11.127083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 607/1188] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:42:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:42:11.131801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17743479-a778-4f02-bce6-c2594f1247f9 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:11.226824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17743479-a778-4f02-bce6-c2594f1247f9 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:11.259366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf7548-ce0e-485c-ae17-458590b0553a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:11.259967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf7548-ce0e-485c-ae17-458590b0553a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:11.260248 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf7548-ce0e-485c-ae17-458590b0553a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:11.260511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf7548-ce0e-485c-ae17-458590b0553a None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:11.261222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf7548-ce0e-485c-ae17-458590b0553a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:42:11.261527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf7548-ce0e-485c-ae17-458590b0553a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:11.262917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf7548-ce0e-485c-ae17-458590b0553a None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:11.270759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 582/1189] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:42:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:42:11.276103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d81b1-0cbb-4e5f-b5d7-c1a91b7c021a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:11.372413 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d81b1-0cbb-4e5f-b5d7-c1a91b7c021a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:11.403711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7c177-9013-4ce7-b47a-b20aa527ed8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:11.404821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7c177-9013-4ce7-b47a-b20aa527ed8a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:11.405145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7c177-9013-4ce7-b47a-b20aa527ed8a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:11.405413 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7c177-9013-4ce7-b47a-b20aa527ed8a None admin] PATH_INFO: `/v3/users/67dae0a48b5443ad86952fc38224d990` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:11.406575 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae7c177-9013-4ce7-b47a-b20aa527ed8a None admin] RBAC: Authorizing `identity:get_user(user_id=67dae0a48b5443ad86952fc38224d990)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:11.407976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae7c177-9013-4ce7-b47a-b20aa527ed8a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:11.409517 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 608/1190] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:42:11 2022] GET /identity/v3/users/67dae0a48b5443ad86952fc38224d990 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:42:11.415005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf7548-ce0e-485c-ae17-458590b0553a None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:11.460042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:11.511847 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf7548-ce0e-485c-ae17-458590b0553a None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:11.543686 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e4a5c-3e5f-4025-a1e7-287ab1913fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:11.544305 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e4a5c-3e5f-4025-a1e7-287ab1913fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:11.544585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e4a5c-3e5f-4025-a1e7-287ab1913fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:11.544859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e4a5c-3e5f-4025-a1e7-287ab1913fb9 None admin] PATH_INFO: `/v3/projects/67440e36640a4835a6f2f77e89a3ee63` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:11.546586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e4a5c-3e5f-4025-a1e7-287ab1913fb9 None admin] RBAC: Authorizing `identity:get_project(project_id=67440e36640a4835a6f2f77e89a3ee63)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:11.548002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e4a5c-3e5f-4025-a1e7-287ab1913fb9 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:11.549636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 583/1191] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:42:11 2022] GET /identity/v3/projects/67440e36640a4835a6f2f77e89a3ee63 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:42:11.555524 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7c177-9013-4ce7-b47a-b20aa527ed8a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:11.601874 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:11.652914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7c177-9013-4ce7-b47a-b20aa527ed8a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:11.685331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9574ef-6360-4d44-92ed-9757ed5e9eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:11.685992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9574ef-6360-4d44-92ed-9757ed5e9eaa None admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:11.686273 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9574ef-6360-4d44-92ed-9757ed5e9eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:11.686579 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9574ef-6360-4d44-92ed-9757ed5e9eaa None admin] PATH_INFO: `/v3/projects/67440e36640a4835a6f2f77e89a3ee63/users/67dae0a48b5443ad86952fc38224d990/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:11.688504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9574ef-6360-4d44-92ed-9757ed5e9eaa None admin] RBAC: Authorizing `identity:create_grant(project_id=67440e36640a4835a6f2f77e89a3ee63, user_id=67dae0a48b5443ad86952fc38224d990, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:11.689915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9574ef-6360-4d44-92ed-9757ed5e9eaa None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:11.702397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 609/1192] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:42:11 2022] PUT /identity/v3/projects/67440e36640a4835a6f2f77e89a3ee63/users/67dae0a48b5443ad86952fc38224d990/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:42:13.628780 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd84e3-9d5a-4ffe-bfaf-5e5ae730814b None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:13.629435 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd84e3-9d5a-4ffe-bfaf-5e5ae730814b None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:13.629435 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd84e3-9d5a-4ffe-bfaf-5e5ae730814b None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:13.630695 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 584/1193] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:42:13.635755 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3342935-7045-4865-b18e-ae603b5663d3 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:13.636194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3342935-7045-4865-b18e-ae603b5663d3 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:13.636318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3342935-7045-4865-b18e-ae603b5663d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:13.653911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-a3342935-7045-4865-b18e-ae603b5663d3 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:42:13.655631 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a3342935-7045-4865-b18e-ae603b5663d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:42:13.719299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 610/1194] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:42:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:42:13.742441 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd84e3-9d5a-4ffe-bfaf-5e5ae730814b None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:13.841502 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd84e3-9d5a-4ffe-bfaf-5e5ae730814b None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:13.855024 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:13.875397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89de43-882d-4ca4-b834-606b3ddd7801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:13.876032 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89de43-882d-4ca4-b834-606b3ddd7801 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:13.876313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89de43-882d-4ca4-b834-606b3ddd7801 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:13.876586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89de43-882d-4ca4-b834-606b3ddd7801 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:13.886987 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89de43-882d-4ca4-b834-606b3ddd7801 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:42:13.892349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-cf89de43-882d-4ca4-b834-606b3ddd7801 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:42:13.892349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:42:13.892349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:42:13.892349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:42:13.892349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:42:13.892349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:42:13.892349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:42:13.892349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:42:13.892349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:42:13.892349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:42:13.892349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:42:13.892349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:42:13.892349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:42:13.892349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:42:13.892349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:42:13.892349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:42:13.893555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:42:13.893555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:42:13.893555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:42:13.893555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:42:13.893555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:42:13.893555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:42:13.893555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:42:13.893555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:42:13.893555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:42:13.893555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:42:13.893555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:42:13.893555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:42:13.893555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:42:13.893555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:42:13.893555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:42:13.893555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:42:13.894528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:42:13.894528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:42:13.894528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:42:13.894528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:42:13.894528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:42:13.894528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:42:13.894528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:42:13.894528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:42:13.894528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:42:13.894528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:42:13.894528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:42:13.894528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:42:13.894528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 585/1195] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:42:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:42:13.899488 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3342935-7045-4865-b18e-ae603b5663d3 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:13.932823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:13.995513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3342935-7045-4865-b18e-ae603b5663d3 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:14.028082 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afea6f8-1860-4da7-964e-a98a8ef92b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:14.028687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afea6f8-1860-4da7-964e-a98a8ef92b79 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:14.028969 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afea6f8-1860-4da7-964e-a98a8ef92b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:14.029260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afea6f8-1860-4da7-964e-a98a8ef92b79 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:14.030597 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afea6f8-1860-4da7-964e-a98a8ef92b79 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:42:14.030913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afea6f8-1860-4da7-964e-a98a8ef92b79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:14.032333 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afea6f8-1860-4da7-964e-a98a8ef92b79 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:14.040073 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 611/1196] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:42:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:42:14.045861 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89de43-882d-4ca4-b834-606b3ddd7801 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:14.141342 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89de43-882d-4ca4-b834-606b3ddd7801 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:14.173394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3c092-5232-495e-9434-37b57743ad01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:14.174013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3c092-5232-495e-9434-37b57743ad01 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:14.174293 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3c092-5232-495e-9434-37b57743ad01 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:14.174591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3c092-5232-495e-9434-37b57743ad01 None admin] PATH_INFO: `/v3/users/67dae0a48b5443ad86952fc38224d990` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:14.175746 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3c092-5232-495e-9434-37b57743ad01 None admin] RBAC: Authorizing `identity:get_user(user_id=67dae0a48b5443ad86952fc38224d990)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:14.177154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3c092-5232-495e-9434-37b57743ad01 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:14.178741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 586/1197] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 3 12:42:14 2022] GET /identity/v3/users/67dae0a48b5443ad86952fc38224d990 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:42:14.184531 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afea6f8-1860-4da7-964e-a98a8ef92b79 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:14.290214 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:14.294146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afea6f8-1860-4da7-964e-a98a8ef92b79 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:14.327894 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be54ea3-c3c5-4b99-b84b-fcacc9fd3726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:14.328499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be54ea3-c3c5-4b99-b84b-fcacc9fd3726 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:14.328781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be54ea3-c3c5-4b99-b84b-fcacc9fd3726 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:14.329057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be54ea3-c3c5-4b99-b84b-fcacc9fd3726 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:14.393382 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be54ea3-c3c5-4b99-b84b-fcacc9fd3726 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:14.394765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be54ea3-c3c5-4b99-b84b-fcacc9fd3726 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:14.484277 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:14.488641 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 612/1198] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:42:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:42:14.494231 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3c092-5232-495e-9434-37b57743ad01 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:14.536427 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:14.592612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3c092-5232-495e-9434-37b57743ad01 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:14.624972 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33007d-6b4a-4ab9-931e-9b8b5e7dc62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:14.625592 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33007d-6b4a-4ab9-931e-9b8b5e7dc62f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:14.625869 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33007d-6b4a-4ab9-931e-9b8b5e7dc62f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:14.626139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33007d-6b4a-4ab9-931e-9b8b5e7dc62f None admin] PATH_INFO: `/v3/projects/67440e36640a4835a6f2f77e89a3ee63` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:14.627305 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33007d-6b4a-4ab9-931e-9b8b5e7dc62f None admin] RBAC: Authorizing `identity:get_project(project_id=67440e36640a4835a6f2f77e89a3ee63)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:14.628719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33007d-6b4a-4ab9-931e-9b8b5e7dc62f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:14.630340 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 587/1199] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 3 12:42:14 2022] GET /identity/v3/projects/67440e36640a4835a6f2f77e89a3ee63 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:42:14.636298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be54ea3-c3c5-4b99-b84b-fcacc9fd3726 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:42:14.729727 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be54ea3-c3c5-4b99-b84b-fcacc9fd3726 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:42:14.762159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3e3322-dfda-4ea0-943e-1071eea31669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:42:14.762776 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e3322-dfda-4ea0-943e-1071eea31669 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:42:14.763052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e3322-dfda-4ea0-943e-1071eea31669 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:42:14.763322 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3e3322-dfda-4ea0-943e-1071eea31669 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:42:14.764115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e3322-dfda-4ea0-943e-1071eea31669 None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=67440e36640a4835a6f2f77e89a3ee63, user.id=67dae0a48b5443ad86952fc38224d990) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:42:14.764420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e3322-dfda-4ea0-943e-1071eea31669 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:42:14.765809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3e3322-dfda-4ea0-943e-1071eea31669 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:42:14.771328 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 613/1200] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:42:14 2022] GET /identity/v3/role_assignments?user.id=67dae0a48b5443ad86952fc38224d990&scope.project.id=67440e36640a4835a6f2f77e89a3ee63&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:42:24.620414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=45 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:42:24.768829 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=69 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:09.647346 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f729464-8ddf-472d-a54f-afa46641a2db None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:09.647950 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f729464-8ddf-472d-a54f-afa46641a2db None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:09.647950 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f729464-8ddf-472d-a54f-afa46641a2db None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:09.649539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 588/1201] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:43:09.655890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5e991-28b0-4e25-9fdf-13e9392e3db7 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:09.656424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5e991-28b0-4e25-9fdf-13e9392e3db7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:09.656424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c5e991-28b0-4e25-9fdf-13e9392e3db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:09.678736 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-f5c5e991-28b0-4e25-9fdf-13e9392e3db7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:43:09.679936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f5c5e991-28b0-4e25-9fdf-13e9392e3db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:43:09.766680 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 614/1202] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:43:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:43:09.786433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f729464-8ddf-472d-a54f-afa46641a2db None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:09.917713 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f729464-8ddf-472d-a54f-afa46641a2db None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:09.968021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e271e3e-21a0-4248-88f7-db2c6bf4d540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:09.968021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e271e3e-21a0-4248-88f7-db2c6bf4d540 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:09.968021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e271e3e-21a0-4248-88f7-db2c6bf4d540 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:09.968021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e271e3e-21a0-4248-88f7-db2c6bf4d540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:09.981792 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e271e3e-21a0-4248-88f7-db2c6bf4d540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:09.983557 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e271e3e-21a0-4248-88f7-db2c6bf4d540 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:10.000015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-6e271e3e-21a0-4248-88f7-db2c6bf4d540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:10.000015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:10.000015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:10.000015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:10.000015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:10.000015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:10.000015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:10.000015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:10.000015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:10.000015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:10.000015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:10.000015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:10.000015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:43:10.000015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:43:10.000015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:10.001854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:10.003118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:43:10.003118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:43:10.003118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:10.003118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:43:10.003118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 589/1203] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:43:10.007675 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5e991-28b0-4e25-9fdf-13e9392e3db7 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:10.129728 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c5e991-28b0-4e25-9fdf-13e9392e3db7 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:10.162883 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:10.170974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc4e06-667d-4b2f-8e6e-fed8d0340fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:10.171527 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc4e06-667d-4b2f-8e6e-fed8d0340fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:10.171808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc4e06-667d-4b2f-8e6e-fed8d0340fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:10.172080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc4e06-667d-4b2f-8e6e-fed8d0340fe5 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:10.172779 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc4e06-667d-4b2f-8e6e-fed8d0340fe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:10.173084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc4e06-667d-4b2f-8e6e-fed8d0340fe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:10.174609 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cc4e06-667d-4b2f-8e6e-fed8d0340fe5 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:10.189099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 615/1204] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:10.195130 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e271e3e-21a0-4248-88f7-db2c6bf4d540 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:10.259118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:10.315640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e271e3e-21a0-4248-88f7-db2c6bf4d540 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:10.365096 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7522b62-615c-4f52-8c11-8c4ea779542f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:10.366129 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7522b62-615c-4f52-8c11-8c4ea779542f None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:10.366407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7522b62-615c-4f52-8c11-8c4ea779542f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:10.366766 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7522b62-615c-4f52-8c11-8c4ea779542f None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:10.367491 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7522b62-615c-4f52-8c11-8c4ea779542f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:10.369075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7522b62-615c-4f52-8c11-8c4ea779542f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:10.398530 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 590/1205] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 3 12:43:10 2022] POST /identity/v3/users => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:43:12.549056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e32d3-d366-46d4-9381-125c78138a51 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:12.549652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e32d3-d366-46d4-9381-125c78138a51 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:12.549652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e32d3-d366-46d4-9381-125c78138a51 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:12.551189 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 616/1206] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:43:12.557040 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255848d-b131-4292-95cf-5db605842ca7 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:12.557778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255848d-b131-4292-95cf-5db605842ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:12.557997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255848d-b131-4292-95cf-5db605842ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:12.581084 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-b255848d-b131-4292-95cf-5db605842ca7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:43:12.582284 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-b255848d-b131-4292-95cf-5db605842ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:43:12.659161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 591/1207] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:43:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:43:12.685281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e32d3-d366-46d4-9381-125c78138a51 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:12.796945 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e32d3-d366-46d4-9381-125c78138a51 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:12.835686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48c0b9-9f50-4158-909c-8414e612ca5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:12.836287 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48c0b9-9f50-4158-909c-8414e612ca5a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:12.836555 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48c0b9-9f50-4158-909c-8414e612ca5a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:12.836815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48c0b9-9f50-4158-909c-8414e612ca5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:12.848281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48c0b9-9f50-4158-909c-8414e612ca5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:43:12.853298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-cc48c0b9-9f50-4158-909c-8414e612ca5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:43:12.853298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:12.853298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:12.853298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:12.853298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:12.853298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:12.853298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:12.853298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:12.853298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:12.853298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:12.853298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:12.853298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:12.853298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:43:12.853298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:43:12.853298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:43:12.853298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:43:12.854717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:43:12.854717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:43:12.854717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:43:12.854717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:43:12.854717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:12.854717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:12.854717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:12.854717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:12.854717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:12.854717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:12.854717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:12.854717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:12.854717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:12.854717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:12.854717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:12.854717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:12.856210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:12.856210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:12.856210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:12.856210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:12.856210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:43:12.856210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:43:12.856210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:43:12.856210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:43:12.856210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:43:12.856210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:43:12.856210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:43:12.856210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:12.856210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 617/1208] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:43:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:12.861305 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255848d-b131-4292-95cf-5db605842ca7 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:12.909336 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:12.977526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255848d-b131-4292-95cf-5db605842ca7 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:13.022529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82420fcf-5348-4ab3-aa52-217582ede9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:13.022529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82420fcf-5348-4ab3-aa52-217582ede9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:13.022529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82420fcf-5348-4ab3-aa52-217582ede9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:13.022529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82420fcf-5348-4ab3-aa52-217582ede9e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:13.022959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82420fcf-5348-4ab3-aa52-217582ede9e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:13.022959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82420fcf-5348-4ab3-aa52-217582ede9e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:13.024513 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82420fcf-5348-4ab3-aa52-217582ede9e6 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:13.039882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 592/1209] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:43:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:13.041094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48c0b9-9f50-4158-909c-8414e612ca5a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:13.099531 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:13.161600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48c0b9-9f50-4158-909c-8414e612ca5a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:13.211266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916299d-f8c2-4a78-a013-b66f14fc666a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:13.211944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916299d-f8c2-4a78-a013-b66f14fc666a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:13.212222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916299d-f8c2-4a78-a013-b66f14fc666a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:13.212516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a916299d-f8c2-4a78-a013-b66f14fc666a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:13.225379 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916299d-f8c2-4a78-a013-b66f14fc666a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:13.227072 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a916299d-f8c2-4a78-a013-b66f14fc666a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:13.243915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-a916299d-f8c2-4a78-a013-b66f14fc666a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:13.243915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:13.243915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:13.243915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:13.243915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:13.243915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:13.243915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:13.243915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:13.243915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:13.243915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:13.243915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:13.243915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:13.243915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:43:13.243915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:43:13.243915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:13.247242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:13.248451 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:43:13.248451 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:43:13.248451 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:13.248451 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:13.248451 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 618/1210] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:43:13.251612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82420fcf-5348-4ab3-aa52-217582ede9e6 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:13.376986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:13.380484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82420fcf-5348-4ab3-aa52-217582ede9e6 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:13.420002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f3a62-9aaa-4139-879f-584fa7680555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:13.420802 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f3a62-9aaa-4139-879f-584fa7680555 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:13.421252 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f3a62-9aaa-4139-879f-584fa7680555 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:13.421660 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f3a62-9aaa-4139-879f-584fa7680555 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:13.422641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f3a62-9aaa-4139-879f-584fa7680555 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:13.423085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f3a62-9aaa-4139-879f-584fa7680555 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:13.425211 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f3a62-9aaa-4139-879f-584fa7680555 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:13.445829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 593/1211] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:13.447265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916299d-f8c2-4a78-a013-b66f14fc666a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:13.557795 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a916299d-f8c2-4a78-a013-b66f14fc666a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:13.560065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:13.606565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efea998-e3fb-416e-8f1d-43219df03f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:13.607268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efea998-e3fb-416e-8f1d-43219df03f5c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:13.607608 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efea998-e3fb-416e-8f1d-43219df03f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:13.607926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efea998-e3fb-416e-8f1d-43219df03f5c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:13.617091 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efea998-e3fb-416e-8f1d-43219df03f5c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:13.619108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efea998-e3fb-416e-8f1d-43219df03f5c None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:13.634426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-1efea998-e3fb-416e-8f1d-43219df03f5c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 12:43:13.634426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:13.634426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:13.634426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:13.634426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:13.634426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:13.634426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:13.634426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:13.634426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:13.634426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:13.634426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:13.634426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:13.634426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:43:13.634426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:43:13.634426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:43:13.634426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:13.635644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:13.637391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:13.637391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:13.637391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:43:13.637391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:43:13.637391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:43:13.637391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:43:13.637391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:43:13.637391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:43:13.637391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 12:43:13.637391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:13.637391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 619/1212] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:13.643332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f3a62-9aaa-4139-879f-584fa7680555 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:13.764717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f3a62-9aaa-4139-879f-584fa7680555 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:13.800951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef89435-35de-4f79-854a-a825dac8266c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:13.801589 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef89435-35de-4f79-854a-a825dac8266c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:13.801873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef89435-35de-4f79-854a-a825dac8266c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:13.802224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef89435-35de-4f79-854a-a825dac8266c None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:13.803089 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef89435-35de-4f79-854a-a825dac8266c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:13.803417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef89435-35de-4f79-854a-a825dac8266c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:13.804969 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef89435-35de-4f79-854a-a825dac8266c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:13.829119 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 594/1213] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 3 12:43:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:13.830610 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efea998-e3fb-416e-8f1d-43219df03f5c None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:13.949234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efea998-e3fb-416e-8f1d-43219df03f5c None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:13.988230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4edc7-b580-4ef8-ad93-85e90c2a426b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:13.988899 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4edc7-b580-4ef8-ad93-85e90c2a426b None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:13.989232 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4edc7-b580-4ef8-ad93-85e90c2a426b None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:13.989516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4edc7-b580-4ef8-ad93-85e90c2a426b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:14.024186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:14.068779 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4edc7-b580-4ef8-ad93-85e90c2a426b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:14.070291 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4edc7-b580-4ef8-ad93-85e90c2a426b None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:14.188446 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 620/1214] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:43:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:43:14.194474 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef89435-35de-4f79-854a-a825dac8266c None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:14.214536 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:14.316230 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef89435-35de-4f79-854a-a825dac8266c None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:14.362530 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8167c-2d17-438b-89d7-54257e965932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:14.363572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8167c-2d17-438b-89d7-54257e965932 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:14.363875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8167c-2d17-438b-89d7-54257e965932 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:14.364143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8167c-2d17-438b-89d7-54257e965932 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:14.378638 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8167c-2d17-438b-89d7-54257e965932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:14.380539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8167c-2d17-438b-89d7-54257e965932 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:14.400493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-bde8167c-2d17-438b-89d7-54257e965932 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:14.400493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:14.400493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:14.400493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:14.400493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:14.400493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:14.400493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:14.400493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:14.400493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:14.400493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:14.400493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:14.400493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:14.400493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:43:14.400493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:43:14.400493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:14.403295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:14.404446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:43:14.404446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:43:14.404446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:14.404446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:43:14.404446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 595/1215] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:43:14.409029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4edc7-b580-4ef8-ad93-85e90c2a426b None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:14.479822 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:14.525243 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4edc7-b580-4ef8-ad93-85e90c2a426b None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:14.566263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5354a2-89fd-48c5-a666-6c0231f9f753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:14.566986 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5354a2-89fd-48c5-a666-6c0231f9f753 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:14.567211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5354a2-89fd-48c5-a666-6c0231f9f753 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:14.567600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5354a2-89fd-48c5-a666-6c0231f9f753 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:14.568195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5354a2-89fd-48c5-a666-6c0231f9f753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:14.568516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5354a2-89fd-48c5-a666-6c0231f9f753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:14.569992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5354a2-89fd-48c5-a666-6c0231f9f753 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:14.586529 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 621/1216] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:14.592360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8167c-2d17-438b-89d7-54257e965932 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:14.679877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:14.720179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8167c-2d17-438b-89d7-54257e965932 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:14.764278 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522db8ec-a6ba-4cce-8021-074414923784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:14.765158 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522db8ec-a6ba-4cce-8021-074414923784 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:14.765612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522db8ec-a6ba-4cce-8021-074414923784 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:14.766072 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522db8ec-a6ba-4cce-8021-074414923784 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:14.773293 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522db8ec-a6ba-4cce-8021-074414923784 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:14.774945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522db8ec-a6ba-4cce-8021-074414923784 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:14.787577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-522db8ec-a6ba-4cce-8021-074414923784 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:43:14.787577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:14.787577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:14.787577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:14.787577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:14.787577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:14.787577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:14.787577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:14.787577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:14.787577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:14.787577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:14.787577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:14.787577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:43:14.787577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:43:14.787577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:14.793636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:14.794859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:43:14.794859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:43:14.794859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:43:14.794859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:43:14.794859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:43:14.794859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:43:14.794859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:43:14.794859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:43:14.794859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 596/1217] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:43:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:43:14.795479 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5354a2-89fd-48c5-a666-6c0231f9f753 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:14.901509 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5354a2-89fd-48c5-a666-6c0231f9f753 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:14.925185 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:14.941724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95a9e28-0164-4191-8957-02b7eb358816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:14.942447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a9e28-0164-4191-8957-02b7eb358816 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:14.942846 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a9e28-0164-4191-8957-02b7eb358816 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:14.943183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95a9e28-0164-4191-8957-02b7eb358816 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:14.944041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a9e28-0164-4191-8957-02b7eb358816 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:14.944465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a9e28-0164-4191-8957-02b7eb358816 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:14.946139 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95a9e28-0164-4191-8957-02b7eb358816 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:14.959986 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 622/1218] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:43:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:14.966228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522db8ec-a6ba-4cce-8021-074414923784 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:15.076521 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522db8ec-a6ba-4cce-8021-074414923784 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:15.106408 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:15.113646 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168466ac-2701-4d32-b946-516640f7dadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:15.114328 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168466ac-2701-4d32-b946-516640f7dadf None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:15.114641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168466ac-2701-4d32-b946-516640f7dadf None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:15.114918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168466ac-2701-4d32-b946-516640f7dadf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:15.115733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168466ac-2701-4d32-b946-516640f7dadf None admin] RBAC: Adding query filter params (role.id=ac03c5aaca774e3196210f3bdb442418, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=5094263ac471481db48783b19192e01d) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:15.116096 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168466ac-2701-4d32-b946-516640f7dadf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:15.117608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168466ac-2701-4d32-b946-516640f7dadf None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:15.124852 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 597/1219] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:43:14 2022] GET /identity/v3/role_assignments?user.id=5094263ac471481db48783b19192e01d&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=ac03c5aaca774e3196210f3bdb442418 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:17.183087 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b91700-589c-4cee-9414-19fd81fed104 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:17.184128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b91700-589c-4cee-9414-19fd81fed104 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:17.184128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b91700-589c-4cee-9414-19fd81fed104 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:17.185571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 623/1220] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:43:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:43:17.191186 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b1d96-2365-40bd-a1b6-f88dd64f532e None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:17.191626 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b1d96-2365-40bd-a1b6-f88dd64f532e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:17.191717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b1d96-2365-40bd-a1b6-f88dd64f532e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:17.211662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-047b1d96-2365-40bd-a1b6-f88dd64f532e None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:43:17.212848 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-047b1d96-2365-40bd-a1b6-f88dd64f532e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:43:17.287021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 598/1221] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:43:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:43:17.311281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b91700-589c-4cee-9414-19fd81fed104 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:17.422384 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b91700-589c-4cee-9414-19fd81fed104 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:17.467797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca6b6d-2183-4aba-9b84-ff7246b386c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:17.468539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca6b6d-2183-4aba-9b84-ff7246b386c8 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:17.468807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca6b6d-2183-4aba-9b84-ff7246b386c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:17.469130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca6b6d-2183-4aba-9b84-ff7246b386c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:17.480233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca6b6d-2183-4aba-9b84-ff7246b386c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:43:17.485447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-63ca6b6d-2183-4aba-9b84-ff7246b386c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:43:17.485447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:17.485447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:17.485447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:17.485447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:17.485447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:17.485447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:17.485447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:17.485447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:17.485447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:17.485447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:17.485447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:17.485447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:43:17.485447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:43:17.485447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:43:17.485447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:43:17.487041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:43:17.487041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:43:17.487041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:43:17.487041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:43:17.487041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:17.487041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:17.487041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:17.487041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:17.487041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:17.487041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:17.487041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:17.487041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:17.487041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:17.487041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:17.487041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:17.487041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:17.488098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:17.488098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:17.488098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:17.488098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:17.488098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:43:17.488098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:43:17.488098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:43:17.488098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:43:17.488098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:43:17.488098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:43:17.488098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:43:17.488098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:17.491978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 624/1222] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:43:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:17.493072 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b1d96-2365-40bd-a1b6-f88dd64f532e None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:17.603935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b1d96-2365-40bd-a1b6-f88dd64f532e None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:17.629353 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:17.643272 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8c017-8fbb-448f-8905-a8b6329be0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:17.643992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8c017-8fbb-448f-8905-a8b6329be0df None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:17.644301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8c017-8fbb-448f-8905-a8b6329be0df None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:17.644595 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8c017-8fbb-448f-8905-a8b6329be0df None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:17.645492 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d8c017-8fbb-448f-8905-a8b6329be0df None admin] RBAC: Adding query filter params (name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:17.645803 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d8c017-8fbb-448f-8905-a8b6329be0df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:17.647265 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d8c017-8fbb-448f-8905-a8b6329be0df None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:17.656579 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 599/1223] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:43:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:17.662339 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca6b6d-2183-4aba-9b84-ff7246b386c8 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:17.712408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:17.776089 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca6b6d-2183-4aba-9b84-ff7246b386c8 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:17.818098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abd91d-d1ad-409a-9201-64fb4b1e15bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:17.818861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abd91d-d1ad-409a-9201-64fb4b1e15bf None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:17.819166 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abd91d-d1ad-409a-9201-64fb4b1e15bf None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:17.819460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0abd91d-d1ad-409a-9201-64fb4b1e15bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:17.830306 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0abd91d-d1ad-409a-9201-64fb4b1e15bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:17.831865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0abd91d-d1ad-409a-9201-64fb4b1e15bf None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:17.847550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-b0abd91d-d1ad-409a-9201-64fb4b1e15bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:17.847550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:17.847550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:17.847550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:17.847550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:17.847550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:17.847550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:17.847550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:17.847550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:17.847550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:17.847550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:17.847550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:17.847550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:43:17.847550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:43:17.847550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:17.848963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:17.849875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:43:17.849875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:43:17.849875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:17.849875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:17.849875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 625/1224] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:17.854927 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8c017-8fbb-448f-8905-a8b6329be0df None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:17.964139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8c017-8fbb-448f-8905-a8b6329be0df None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:17.998812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9a162-0619-4a00-ae8b-dc6223b0438c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:17.999528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9a162-0619-4a00-ae8b-dc6223b0438c None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:17.999886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9a162-0619-4a00-ae8b-dc6223b0438c None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:18.000352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9a162-0619-4a00-ae8b-dc6223b0438c None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:18.001248 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9a162-0619-4a00-ae8b-dc6223b0438c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:18.001559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9a162-0619-4a00-ae8b-dc6223b0438c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:18.003204 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e9a162-0619-4a00-ae8b-dc6223b0438c None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:18.015306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 600/1225] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:18.021005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abd91d-d1ad-409a-9201-64fb4b1e15bf None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:18.120477 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:18.127994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0abd91d-d1ad-409a-9201-64fb4b1e15bf None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:18.170585 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f5a7f-1bdf-4224-9d47-9bb75fa19b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:18.171485 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f5a7f-1bdf-4224-9d47-9bb75fa19b1d None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:18.171942 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f5a7f-1bdf-4224-9d47-9bb75fa19b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:18.172471 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f5a7f-1bdf-4224-9d47-9bb75fa19b1d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:18.182610 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f5a7f-1bdf-4224-9d47-9bb75fa19b1d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:18.184057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f5a7f-1bdf-4224-9d47-9bb75fa19b1d None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:18.195006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-6f3f5a7f-1bdf-4224-9d47-9bb75fa19b1d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 12:43:18.195006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:18.195006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:18.195006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:18.195006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:18.195006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:18.195006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:18.195006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:18.195006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:18.195006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:18.195006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:18.195006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:18.195006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:43:18.195006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:43:18.195006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:43:18.195006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:18.196135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:18.197515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:18.197515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:18.197515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:43:18.197515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:43:18.197515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:43:18.197515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:43:18.197515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:43:18.197515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:43:18.197515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 12:43:18.197515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:18.197515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 626/1226] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:18.201977 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9a162-0619-4a00-ae8b-dc6223b0438c None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:18.255489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:18.311535 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9a162-0619-4a00-ae8b-dc6223b0438c None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:18.350257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341587a-0cbd-46d3-a2a0-f413115cc59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:18.351033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341587a-0cbd-46d3-a2a0-f413115cc59f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:18.351382 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341587a-0cbd-46d3-a2a0-f413115cc59f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:18.351774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341587a-0cbd-46d3-a2a0-f413115cc59f None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:18.352737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341587a-0cbd-46d3-a2a0-f413115cc59f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:18.353190 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341587a-0cbd-46d3-a2a0-f413115cc59f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:18.354762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341587a-0cbd-46d3-a2a0-f413115cc59f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:18.379788 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 601/1227] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 3 12:43:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:18.381028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f5a7f-1bdf-4224-9d47-9bb75fa19b1d None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:18.483648 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f5a7f-1bdf-4224-9d47-9bb75fa19b1d None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:18.518902 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07d6de-b156-4841-a0f9-af7fe0e7e756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:18.519505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07d6de-b156-4841-a0f9-af7fe0e7e756 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:18.519789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07d6de-b156-4841-a0f9-af7fe0e7e756 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:18.520063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07d6de-b156-4841-a0f9-af7fe0e7e756 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:18.531863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07d6de-b156-4841-a0f9-af7fe0e7e756 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:18.533301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07d6de-b156-4841-a0f9-af7fe0e7e756 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:18.548537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-cb07d6de-b156-4841-a0f9-af7fe0e7e756 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:18.548537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:18.548537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:18.548537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:18.548537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:18.548537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:18.548537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:18.548537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:18.548537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:18.548537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:18.548537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:18.548537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:18.548537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:43:18.548537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:43:18.548537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:18.555420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:18.556587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:43:18.556587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:43:18.556587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:18.556587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:18.556587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 627/1228] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:18.556946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341587a-0cbd-46d3-a2a0-f413115cc59f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:18.680219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341587a-0cbd-46d3-a2a0-f413115cc59f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:18.683966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:18.719843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f8e8d-67e3-4754-b420-00f88ef7aad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:18.720460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f8e8d-67e3-4754-b420-00f88ef7aad9 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:18.720738 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f8e8d-67e3-4754-b420-00f88ef7aad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:18.721007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f8e8d-67e3-4754-b420-00f88ef7aad9 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:18.721697 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f8e8d-67e3-4754-b420-00f88ef7aad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:18.721996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f8e8d-67e3-4754-b420-00f88ef7aad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:18.723407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f8e8d-67e3-4754-b420-00f88ef7aad9 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:18.735422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 602/1229] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:18.741604 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07d6de-b156-4841-a0f9-af7fe0e7e756 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:18.752866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:18.850600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07d6de-b156-4841-a0f9-af7fe0e7e756 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:18.886344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35488d-1bd4-468d-b056-031e78daa527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:18.887579 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35488d-1bd4-468d-b056-031e78daa527 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:18.887855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35488d-1bd4-468d-b056-031e78daa527 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:18.888126 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e35488d-1bd4-468d-b056-031e78daa527 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:18.899001 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35488d-1bd4-468d-b056-031e78daa527 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:18.900442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e35488d-1bd4-468d-b056-031e78daa527 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:18.913021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-0e35488d-1bd4-468d-b056-031e78daa527 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:43:18.913021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:18.913021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:18.913021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:18.913021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:18.913021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:18.913021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:18.913021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:18.913021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:18.913021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:18.913021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:18.913021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:18.913021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:43:18.913021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:43:18.913021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:18.914128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:18.916467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:43:18.916467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:43:18.916467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:43:18.916467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:43:18.916467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:43:18.916467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:43:18.916467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:43:18.916467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:18.916467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 628/1230] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:43:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:18.920338 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f8e8d-67e3-4754-b420-00f88ef7aad9 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:19.032372 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f8e8d-67e3-4754-b420-00f88ef7aad9 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:19.084423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90eb23-8f92-4a1e-aad7-7f3ca141b602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:19.085433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90eb23-8f92-4a1e-aad7-7f3ca141b602 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:19.085909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90eb23-8f92-4a1e-aad7-7f3ca141b602 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:19.086382 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90eb23-8f92-4a1e-aad7-7f3ca141b602 None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:19.087966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e90eb23-8f92-4a1e-aad7-7f3ca141b602 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:19.088545 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e90eb23-8f92-4a1e-aad7-7f3ca141b602 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:19.091557 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e90eb23-8f92-4a1e-aad7-7f3ca141b602 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:19.115272 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 603/1231] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:43:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:19.125497 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35488d-1bd4-468d-b056-031e78daa527 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:19.203978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:19.242809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e35488d-1bd4-468d-b056-031e78daa527 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:19.277517 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919a85e-31e4-4243-b2de-4c01804b5822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:19.278171 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919a85e-31e4-4243-b2de-4c01804b5822 None admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:19.278441 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919a85e-31e4-4243-b2de-4c01804b5822 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:19.278752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919a85e-31e4-4243-b2de-4c01804b5822 None admin] PATH_INFO: `/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/5094263ac471481db48783b19192e01d/roles/ac03c5aaca774e3196210f3bdb442418` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:19.293669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919a85e-31e4-4243-b2de-4c01804b5822 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0774035ad0f41c986771ef4a41c7d60, user_id=5094263ac471481db48783b19192e01d, role_id=ac03c5aaca774e3196210f3bdb442418)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:19.295061 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919a85e-31e4-4243-b2de-4c01804b5822 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:19.309614 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 629/1232] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:43:19 2022] PUT /identity/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/5094263ac471481db48783b19192e01d/roles/ac03c5aaca774e3196210f3bdb442418 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:43:21.506417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d708495-747a-4772-b831-12442c00f1eb None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:21.507110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d708495-747a-4772-b831-12442c00f1eb None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:21.507223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d708495-747a-4772-b831-12442c00f1eb None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:21.508558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 604/1233] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:43:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:43:21.515327 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044995ea-7344-43ee-b922-790c2cea0239 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:21.515763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044995ea-7344-43ee-b922-790c2cea0239 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:21.515869 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044995ea-7344-43ee-b922-790c2cea0239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:21.534250 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-044995ea-7344-43ee-b922-790c2cea0239 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:43:21.535868 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-044995ea-7344-43ee-b922-790c2cea0239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:43:21.619202 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 630/1234] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:43:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:43:21.642292 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d708495-747a-4772-b831-12442c00f1eb None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:21.668408 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:21.748076 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d708495-747a-4772-b831-12442c00f1eb None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:21.782570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dc5a2-92d8-4451-a9ed-c51dbba5856b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:21.783161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dc5a2-92d8-4451-a9ed-c51dbba5856b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:21.783444 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dc5a2-92d8-4451-a9ed-c51dbba5856b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:21.783713 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6dc5a2-92d8-4451-a9ed-c51dbba5856b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:21.794468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6dc5a2-92d8-4451-a9ed-c51dbba5856b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:43:21.800036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-4b6dc5a2-92d8-4451-a9ed-c51dbba5856b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:43:21.800036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:21.800036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:21.800036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:21.800036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:21.800036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:21.800036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:21.800036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:21.800036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:21.800036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:21.800036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:21.800036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:21.800036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:43:21.800036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:43:21.800036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:43:21.800036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:43:21.801002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:43:21.801002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:43:21.801002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:43:21.801002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:43:21.801002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:21.801002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:21.801002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:21.801002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:21.801002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:21.801002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:21.801002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:21.801002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:21.801002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:21.801002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:21.801002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:21.801002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:21.802598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:21.802598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:21.802598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:21.802598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:21.802598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:43:21.802598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:43:21.802598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:43:21.802598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:43:21.802598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:43:21.802598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:43:21.802598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 03 12:43:21.802598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:43:21.802598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 605/1235] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 3 12:43:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:21.807872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044995ea-7344-43ee-b922-790c2cea0239 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:21.856584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:21.914559 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044995ea-7344-43ee-b922-790c2cea0239 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:21.952695 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93218c2f-04ad-4cd8-b5dd-d67e7315aa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:21.953308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93218c2f-04ad-4cd8-b5dd-d67e7315aa04 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:21.953584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93218c2f-04ad-4cd8-b5dd-d67e7315aa04 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:21.953851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93218c2f-04ad-4cd8-b5dd-d67e7315aa04 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:21.954585 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93218c2f-04ad-4cd8-b5dd-d67e7315aa04 None admin] RBAC: Adding query filter params (name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:21.954887 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93218c2f-04ad-4cd8-b5dd-d67e7315aa04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:21.956233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93218c2f-04ad-4cd8-b5dd-d67e7315aa04 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:21.964777 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 631/1236] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 3 12:43:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:21.970280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dc5a2-92d8-4451-a9ed-c51dbba5856b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:22.059554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:22.078211 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6dc5a2-92d8-4451-a9ed-c51dbba5856b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:22.114149 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf700dbe-13da-409c-858a-645de3e55f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:22.114998 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf700dbe-13da-409c-858a-645de3e55f0d None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:22.115280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf700dbe-13da-409c-858a-645de3e55f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:22.115692 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf700dbe-13da-409c-858a-645de3e55f0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:22.127464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf700dbe-13da-409c-858a-645de3e55f0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:22.128924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf700dbe-13da-409c-858a-645de3e55f0d None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:22.144875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-bf700dbe-13da-409c-858a-645de3e55f0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:22.144875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:22.144875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:22.144875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:22.144875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:22.144875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:22.144875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:22.144875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:22.144875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:22.144875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:22.144875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:22.144875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:22.144875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:43:22.144875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:43:22.144875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:22.146378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:22.147588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:43:22.147588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:43:22.147588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:22.147588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:43:22.147588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 606/1237] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:22.152596 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93218c2f-04ad-4cd8-b5dd-d67e7315aa04 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:22.263094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93218c2f-04ad-4cd8-b5dd-d67e7315aa04 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:22.272154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:22.298472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd504779-5642-4623-95dd-4fb3819627cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:22.299115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd504779-5642-4623-95dd-4fb3819627cf None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:22.299397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd504779-5642-4623-95dd-4fb3819627cf None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:22.299667 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd504779-5642-4623-95dd-4fb3819627cf None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:22.300358 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd504779-5642-4623-95dd-4fb3819627cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:22.300662 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd504779-5642-4623-95dd-4fb3819627cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:22.302017 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd504779-5642-4623-95dd-4fb3819627cf None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:22.314744 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 632/1238] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:22.320460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf700dbe-13da-409c-858a-645de3e55f0d None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:22.430683 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf700dbe-13da-409c-858a-645de3e55f0d None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:22.471871 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799a8a8-a1cb-47cd-b76f-705960dd7778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:22.472688 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799a8a8-a1cb-47cd-b76f-705960dd7778 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:22.473054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799a8a8-a1cb-47cd-b76f-705960dd7778 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:22.473421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799a8a8-a1cb-47cd-b76f-705960dd7778 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:22.477232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:22.487039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799a8a8-a1cb-47cd-b76f-705960dd7778 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:22.489034 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8799a8a8-a1cb-47cd-b76f-705960dd7778 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:22.502303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-8799a8a8-a1cb-47cd-b76f-705960dd7778 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 12:43:22.502303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:22.502303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:22.502303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:22.502303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:22.502303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:22.502303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:22.502303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:22.502303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:22.502303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:22.502303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:22.502303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:22.502303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:43:22.502303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:43:22.502303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:43:22.502303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:22.503578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:22.504529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:22.504529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:22.504529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:43:22.504529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:43:22.504529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:43:22.504529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:43:22.504529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:43:22.504529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:43:22.504529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 12:43:22.504529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:43:22.504529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 607/1239] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:22.509508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd504779-5642-4623-95dd-4fb3819627cf None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:22.619278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd504779-5642-4623-95dd-4fb3819627cf None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:22.662634 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97255e60-2540-45fd-a42c-79412a6c6ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:22.663260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97255e60-2540-45fd-a42c-79412a6c6ded None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:22.663537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97255e60-2540-45fd-a42c-79412a6c6ded None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:22.663809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97255e60-2540-45fd-a42c-79412a6c6ded None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:22.664603 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97255e60-2540-45fd-a42c-79412a6c6ded None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:22.664908 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97255e60-2540-45fd-a42c-79412a6c6ded None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:22.666331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97255e60-2540-45fd-a42c-79412a6c6ded None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:22.685806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 633/1240] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 3 12:43:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:22.692165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799a8a8-a1cb-47cd-b76f-705960dd7778 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:22.802407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799a8a8-a1cb-47cd-b76f-705960dd7778 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:22.838839 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb2693-b2a1-4129-a0a0-115c33bc4bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:22.839453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb2693-b2a1-4129-a0a0-115c33bc4bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:22.840254 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb2693-b2a1-4129-a0a0-115c33bc4bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:22.840523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb2693-b2a1-4129-a0a0-115c33bc4bf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:22.914403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:22.922033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb2693-b2a1-4129-a0a0-115c33bc4bf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:22.923697 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb2693-b2a1-4129-a0a0-115c33bc4bf8 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:23.039085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 608/1241] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:43:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:43:23.044838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97255e60-2540-45fd-a42c-79412a6c6ded None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:23.078728 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:23.159175 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97255e60-2540-45fd-a42c-79412a6c6ded None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:23.194428 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f245c-cab3-4de2-8c0a-197a1105c4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:23.195100 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f245c-cab3-4de2-8c0a-197a1105c4ae None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:23.195379 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f245c-cab3-4de2-8c0a-197a1105c4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:23.195653 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f245c-cab3-4de2-8c0a-197a1105c4ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:23.206732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f245c-cab3-4de2-8c0a-197a1105c4ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:23.208166 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f245c-cab3-4de2-8c0a-197a1105c4ae None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:23.223687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-ab2f245c-cab3-4de2-8c0a-197a1105c4ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:23.223687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:23.223687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:23.223687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:23.223687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:23.223687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:23.223687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:23.223687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:23.223687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:23.223687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:23.223687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:23.223687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:23.223687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:43:23.223687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:43:23.223687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:23.225067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:23.226159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:43:23.226159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:43:23.226159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:23.226159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:23.226159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 634/1242] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:23.232543 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb2693-b2a1-4129-a0a0-115c33bc4bf8 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:23.336013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:23.341217 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb2693-b2a1-4129-a0a0-115c33bc4bf8 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:23.382290 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f395f-6fca-4462-952f-4622d1fd3740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:23.383011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f395f-6fca-4462-952f-4622d1fd3740 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:23.383241 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f395f-6fca-4462-952f-4622d1fd3740 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:23.383542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f395f-6fca-4462-952f-4622d1fd3740 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:23.384297 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f395f-6fca-4462-952f-4622d1fd3740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:23.384599 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f395f-6fca-4462-952f-4622d1fd3740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:23.386077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f395f-6fca-4462-952f-4622d1fd3740 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:23.400127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 609/1243] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:23.406039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f245c-cab3-4de2-8c0a-197a1105c4ae None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:23.490065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:23.513896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f245c-cab3-4de2-8c0a-197a1105c4ae None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:23.549914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68ebb3-b49a-440f-87fe-167b24ec0894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:23.550581 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68ebb3-b49a-440f-87fe-167b24ec0894 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:23.550863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68ebb3-b49a-440f-87fe-167b24ec0894 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:23.551134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68ebb3-b49a-440f-87fe-167b24ec0894 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:23.559065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed68ebb3-b49a-440f-87fe-167b24ec0894 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:23.560650 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed68ebb3-b49a-440f-87fe-167b24ec0894 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:23.572498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-ed68ebb3-b49a-440f-87fe-167b24ec0894 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:43:23.572498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:23.572498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:23.572498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:23.572498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:23.572498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:23.572498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:23.572498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:23.572498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:23.572498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:23.572498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:23.572498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:23.572498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:43:23.572498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:43:23.572498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:23.573567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:23.574738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:43:23.574738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:43:23.574738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:43:23.574738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:43:23.574738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:43:23.574738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:43:23.574738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:43:23.574738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:23.581626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 635/1244] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:43:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:23.583290 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f395f-6fca-4462-952f-4622d1fd3740 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:23.705122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f395f-6fca-4462-952f-4622d1fd3740 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:23.743010 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de605b-0f02-4cfc-b28f-c3c7dfa45b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:23.743669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de605b-0f02-4cfc-b28f-c3c7dfa45b3f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:23.744060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de605b-0f02-4cfc-b28f-c3c7dfa45b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:23.744399 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de605b-0f02-4cfc-b28f-c3c7dfa45b3f None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:23.745297 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de605b-0f02-4cfc-b28f-c3c7dfa45b3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:23.745621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de605b-0f02-4cfc-b28f-c3c7dfa45b3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:23.748600 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de605b-0f02-4cfc-b28f-c3c7dfa45b3f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:23.762280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 610/1245] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:43:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:23.768443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68ebb3-b49a-440f-87fe-167b24ec0894 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:23.881799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68ebb3-b49a-440f-87fe-167b24ec0894 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:23.918354 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0c890-ccb5-412e-9549-5dead020cca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:23.919085 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0c890-ccb5-412e-9549-5dead020cca8 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:23.919402 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0c890-ccb5-412e-9549-5dead020cca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:23.919752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0c890-ccb5-412e-9549-5dead020cca8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:23.920626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0c890-ccb5-412e-9549-5dead020cca8 None admin] RBAC: Adding query filter params (role.id=ac03c5aaca774e3196210f3bdb442418, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=5094263ac471481db48783b19192e01d) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:23.920938 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0c890-ccb5-412e-9549-5dead020cca8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:23.922437 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0c890-ccb5-412e-9549-5dead020cca8 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:23.929468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 636/1246] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:43:23 2022] GET /identity/v3/role_assignments?user.id=5094263ac471481db48783b19192e01d&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=ac03c5aaca774e3196210f3bdb442418 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:26.391305 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9385d-ccca-4c02-af1a-bc18c8986746 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:26.391882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9385d-ccca-4c02-af1a-bc18c8986746 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:26.391882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9385d-ccca-4c02-af1a-bc18c8986746 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:26.393219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 611/1247] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:43:26.399299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d281988-dc85-4738-9137-4bdd36966702 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:26.399787 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d281988-dc85-4738-9137-4bdd36966702 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:26.399988 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d281988-dc85-4738-9137-4bdd36966702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:26.406995 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:26.420392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1d281988-dc85-4738-9137-4bdd36966702 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:43:26.421700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1d281988-dc85-4738-9137-4bdd36966702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:43:26.499610 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 637/1248] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:43:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 12:43:26.524300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9385d-ccca-4c02-af1a-bc18c8986746 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:26.547861 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:26.649450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9385d-ccca-4c02-af1a-bc18c8986746 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:26.689402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bf679-aed1-496d-bc04-f3108b352ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:26.690062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bf679-aed1-496d-bc04-f3108b352ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:26.690357 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bf679-aed1-496d-bc04-f3108b352ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:26.690809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bf679-aed1-496d-bc04-f3108b352ac5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:26.703179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9bf679-aed1-496d-bc04-f3108b352ac5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:43:26.709139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-3b9bf679-aed1-496d-bc04-f3108b352ac5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:43:26.709139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:26.709139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:26.709139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:26.709139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:26.709139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:26.709139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:26.709139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:26.709139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:26.709139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:26.709139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:26.709139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:26.709139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:43:26.709139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:43:26.709139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:43:26.709139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:43:26.710266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:43:26.710266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:43:26.710266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:43:26.710266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:43:26.710266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:26.710266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:26.710266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:26.710266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:26.710266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:26.710266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:26.710266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:26.710266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:26.710266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:26.710266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:26.710266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:26.710266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:26.711516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:26.711516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:26.711516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:26.711516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:26.711516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:43:26.711516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:43:26.711516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:43:26.711516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:43:26.711516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:43:26.711516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:43:26.711516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:43:26.711516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:43:26.711516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 612/1249] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:43:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:26.717763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d281988-dc85-4738-9137-4bdd36966702 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:26.838212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d281988-dc85-4738-9137-4bdd36966702 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:26.873135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:26.880782 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228e825-8f3a-456d-bec9-9384d1c88b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:26.881440 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228e825-8f3a-456d-bec9-9384d1c88b67 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:26.881769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228e825-8f3a-456d-bec9-9384d1c88b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:26.882066 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228e825-8f3a-456d-bec9-9384d1c88b67 None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:26.882962 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228e825-8f3a-456d-bec9-9384d1c88b67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:26.883297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228e825-8f3a-456d-bec9-9384d1c88b67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:26.884868 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228e825-8f3a-456d-bec9-9384d1c88b67 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:26.895842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 638/1250] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:43:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:26.903495 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bf679-aed1-496d-bc04-f3108b352ac5 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:26.999705 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:27.026317 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bf679-aed1-496d-bc04-f3108b352ac5 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:27.063088 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b05431-28b6-4aa4-bc7e-46473353afe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:27.063743 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b05431-28b6-4aa4-bc7e-46473353afe6 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:27.064023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b05431-28b6-4aa4-bc7e-46473353afe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:27.064296 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b05431-28b6-4aa4-bc7e-46473353afe6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:27.076421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b05431-28b6-4aa4-bc7e-46473353afe6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:27.077988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b05431-28b6-4aa4-bc7e-46473353afe6 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:27.092845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-07b05431-28b6-4aa4-bc7e-46473353afe6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:27.092845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:27.092845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:27.092845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:27.092845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:27.092845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:27.092845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:27.092845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:27.092845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:27.092845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:27.092845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:27.092845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:27.092845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:43:27.092845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:43:27.092845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:27.094228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:27.095454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:43:27.095454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:43:27.095454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:27.095454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:43:27.095454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 613/1251] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:27.100715 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228e825-8f3a-456d-bec9-9384d1c88b67 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:27.212570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228e825-8f3a-456d-bec9-9384d1c88b67 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:27.247836 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ce665-5561-43d6-8f41-2c52fc56014c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:27.248455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ce665-5561-43d6-8f41-2c52fc56014c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:27.248738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ce665-5561-43d6-8f41-2c52fc56014c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:27.249013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ce665-5561-43d6-8f41-2c52fc56014c None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:27.249730 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ce665-5561-43d6-8f41-2c52fc56014c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:27.250639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ce665-5561-43d6-8f41-2c52fc56014c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:27.252082 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ce665-5561-43d6-8f41-2c52fc56014c None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:27.264540 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 639/1252] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:27.270345 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b05431-28b6-4aa4-bc7e-46473353afe6 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:27.385532 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b05431-28b6-4aa4-bc7e-46473353afe6 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:27.412782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:27.421754 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7952986-52fd-4b8d-9f3e-b77ee317419f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:27.422397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7952986-52fd-4b8d-9f3e-b77ee317419f None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:27.422704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7952986-52fd-4b8d-9f3e-b77ee317419f None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:27.422974 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7952986-52fd-4b8d-9f3e-b77ee317419f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:27.431631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7952986-52fd-4b8d-9f3e-b77ee317419f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:27.433900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7952986-52fd-4b8d-9f3e-b77ee317419f None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:27.445451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-f7952986-52fd-4b8d-9f3e-b77ee317419f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 12:43:27.445451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:27.445451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:27.445451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:27.445451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:27.445451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:27.445451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:27.445451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:27.445451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:27.445451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:27.445451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:27.445451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:27.445451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:43:27.445451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:43:27.445451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:43:27.445451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:27.446407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:27.447406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:27.447406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:27.447406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:43:27.447406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:43:27.447406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:43:27.447406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:43:27.447406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:43:27.447406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:43:27.447406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 12:43:27.447406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:43:27.447406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 614/1253] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 03 12:43:27.452636 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ce665-5561-43d6-8f41-2c52fc56014c None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:27.515154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:27.568478 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ce665-5561-43d6-8f41-2c52fc56014c None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:27.610273 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620eaa40-5308-48ff-84c2-21ecb2db6413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:27.610915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620eaa40-5308-48ff-84c2-21ecb2db6413 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:27.611223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620eaa40-5308-48ff-84c2-21ecb2db6413 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:27.611469 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620eaa40-5308-48ff-84c2-21ecb2db6413 None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:27.612257 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620eaa40-5308-48ff-84c2-21ecb2db6413 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:27.612575 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620eaa40-5308-48ff-84c2-21ecb2db6413 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:27.613967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620eaa40-5308-48ff-84c2-21ecb2db6413 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:27.633267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 640/1254] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 3 12:43:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:27.640142 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7952986-52fd-4b8d-9f3e-b77ee317419f None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:27.758103 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7952986-52fd-4b8d-9f3e-b77ee317419f None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:27.797527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9ee8f-4467-4160-951f-7c2e18610715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:27.798115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9ee8f-4467-4160-951f-7c2e18610715 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:27.798579 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9ee8f-4467-4160-951f-7c2e18610715 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:27.798868 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9ee8f-4467-4160-951f-7c2e18610715 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:27.865457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:27.879382 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9ee8f-4467-4160-951f-7c2e18610715 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:27.880841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f9ee8f-4467-4160-951f-7c2e18610715 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:27.983151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 615/1255] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:43:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:43:27.989861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620eaa40-5308-48ff-84c2-21ecb2db6413 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:28.095952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620eaa40-5308-48ff-84c2-21ecb2db6413 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:28.112362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:28.135363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09d605-bcf8-47eb-be97-5ed4e43c7ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:28.136114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09d605-bcf8-47eb-be97-5ed4e43c7ddf None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:28.136877 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09d605-bcf8-47eb-be97-5ed4e43c7ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:28.137165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09d605-bcf8-47eb-be97-5ed4e43c7ddf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:28.147606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af09d605-bcf8-47eb-be97-5ed4e43c7ddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:28.149081 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af09d605-bcf8-47eb-be97-5ed4e43c7ddf None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:28.166248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-af09d605-bcf8-47eb-be97-5ed4e43c7ddf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:28.166248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:28.166248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:28.166248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:28.166248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:28.166248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:28.166248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:28.166248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:28.166248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:28.166248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:28.166248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:28.166248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:28.166248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:43:28.166248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:43:28.166248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:28.167318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:28.173233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:43:28.173233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:43:28.173233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:28.173233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:28.173233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 641/1256] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:28.173601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9ee8f-4467-4160-951f-7c2e18610715 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:28.263770 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:28.280307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9ee8f-4467-4160-951f-7c2e18610715 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:28.321568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24fae1-729d-4a25-a899-52e9e444c8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:28.322238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24fae1-729d-4a25-a899-52e9e444c8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:28.322545 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24fae1-729d-4a25-a899-52e9e444c8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:28.322884 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24fae1-729d-4a25-a899-52e9e444c8c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:28.323582 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24fae1-729d-4a25-a899-52e9e444c8c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:28.324430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24fae1-729d-4a25-a899-52e9e444c8c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:28.325857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24fae1-729d-4a25-a899-52e9e444c8c4 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:28.337782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 616/1257] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:43:28.344684 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09d605-bcf8-47eb-be97-5ed4e43c7ddf None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:28.463025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09d605-bcf8-47eb-be97-5ed4e43c7ddf None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:28.501724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f34dc-01de-4a5e-8fdd-92b217c36015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:28.502371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f34dc-01de-4a5e-8fdd-92b217c36015 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:28.502683 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f34dc-01de-4a5e-8fdd-92b217c36015 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:28.502961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f34dc-01de-4a5e-8fdd-92b217c36015 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:28.510117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f34dc-01de-4a5e-8fdd-92b217c36015 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:28.512492 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f34dc-01de-4a5e-8fdd-92b217c36015 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:28.527503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-241f34dc-01de-4a5e-8fdd-92b217c36015 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:43:28.527503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:28.527503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:28.527503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:28.527503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:28.527503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:28.527503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:28.527503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:28.527503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:28.527503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:28.527503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:28.527503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:28.527503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:43:28.527503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:43:28.527503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:28.528605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:28.529649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:43:28.529649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:43:28.529649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:43:28.529649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:43:28.529649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:43:28.529649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:43:28.529649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:43:28.529649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:28.529649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 642/1258] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:43:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:28.535681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24fae1-729d-4a25-a899-52e9e444c8c4 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:28.663759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24fae1-729d-4a25-a899-52e9e444c8c4 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:28.699701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697d4188-ec3a-43c8-a3fe-bc5bb49b22b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:28.700331 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d4188-ec3a-43c8-a3fe-bc5bb49b22b4 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:28.700607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d4188-ec3a-43c8-a3fe-bc5bb49b22b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:28.700873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697d4188-ec3a-43c8-a3fe-bc5bb49b22b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:28.701622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d4188-ec3a-43c8-a3fe-bc5bb49b22b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:28.701925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d4188-ec3a-43c8-a3fe-bc5bb49b22b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:28.703494 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697d4188-ec3a-43c8-a3fe-bc5bb49b22b4 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:28.716043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 617/1259] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:43:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:28.722166 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f34dc-01de-4a5e-8fdd-92b217c36015 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:28.749185 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:28.830720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f34dc-01de-4a5e-8fdd-92b217c36015 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:28.871207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185887d7-15aa-436f-9b8d-3b71b557a677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:28.871841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185887d7-15aa-436f-9b8d-3b71b557a677 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:28.872119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185887d7-15aa-436f-9b8d-3b71b557a677 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:28.872473 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185887d7-15aa-436f-9b8d-3b71b557a677 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:28.873331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185887d7-15aa-436f-9b8d-3b71b557a677 None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=5094263ac471481db48783b19192e01d) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:28.873586 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185887d7-15aa-436f-9b8d-3b71b557a677 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:28.875048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185887d7-15aa-436f-9b8d-3b71b557a677 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:28.881380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 643/1260] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:43:28 2022] GET /identity/v3/role_assignments?user.id=5094263ac471481db48783b19192e01d&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:30.877910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5348f5-77b7-4485-b1ac-115d0f4f4aa5 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:30.878361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5348f5-77b7-4485-b1ac-115d0f4f4aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:30.878476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5348f5-77b7-4485-b1ac-115d0f4f4aa5 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:30.879853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 618/1261] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:43:30.884961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0aa945-e85e-4785-99dc-02ca74e64d7e None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:30.885360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0aa945-e85e-4785-99dc-02ca74e64d7e None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:30.885511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0aa945-e85e-4785-99dc-02ca74e64d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:30.903628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-dc0aa945-e85e-4785-99dc-02ca74e64d7e None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:43:30.904804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-dc0aa945-e85e-4785-99dc-02ca74e64d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:43:30.968107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 644/1262] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:43:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 12:43:30.990572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5348f5-77b7-4485-b1ac-115d0f4f4aa5 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:31.001187 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:31.090615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5348f5-77b7-4485-b1ac-115d0f4f4aa5 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:31.122554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae1ed1-2a93-46a2-91dc-2891f8900f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:31.123184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae1ed1-2a93-46a2-91dc-2891f8900f8a None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:31.123464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae1ed1-2a93-46a2-91dc-2891f8900f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:31.123737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae1ed1-2a93-46a2-91dc-2891f8900f8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:31.133496 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae1ed1-2a93-46a2-91dc-2891f8900f8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:43:31.138824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-79ae1ed1-2a93-46a2-91dc-2891f8900f8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:43:31.138824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:31.138824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:31.138824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:31.138824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:31.138824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:31.138824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:31.138824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:31.138824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:31.138824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:31.138824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:31.138824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:31.138824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:43:31.138824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:43:31.138824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:43:31.138824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:43:31.139913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:43:31.139913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:43:31.139913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:43:31.139913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:43:31.139913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:31.139913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:31.139913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:31.139913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:31.139913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:31.139913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:31.139913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:31.139913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:31.139913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:31.139913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:31.139913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:31.139913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:31.140910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:31.140910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:31.140910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:31.140910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:31.140910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:43:31.140910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:43:31.140910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:43:31.140910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:43:31.140910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:43:31.140910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:43:31.140910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:43:31.140910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:43:31.140910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 619/1263] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:43:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:31.145756 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0aa945-e85e-4785-99dc-02ca74e64d7e None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:31.163886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:31.240760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0aa945-e85e-4785-99dc-02ca74e64d7e None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:31.272693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ad09d-9a24-4d47-89f2-6bc15d98d6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:31.273286 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ad09d-9a24-4d47-89f2-6bc15d98d6cb None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:31.273562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ad09d-9a24-4d47-89f2-6bc15d98d6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:31.273828 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ad09d-9a24-4d47-89f2-6bc15d98d6cb None admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:31.274546 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ad09d-9a24-4d47-89f2-6bc15d98d6cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:31.274849 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ad09d-9a24-4d47-89f2-6bc15d98d6cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:31.276197 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ad09d-9a24-4d47-89f2-6bc15d98d6cb None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:31.284473 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 645/1264] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:43:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:31.289976 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae1ed1-2a93-46a2-91dc-2891f8900f8a None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:31.355193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:31.387722 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae1ed1-2a93-46a2-91dc-2891f8900f8a None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:31.420514 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6a2ca-5736-4051-a9a8-8435c5d5020b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:31.421133 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6a2ca-5736-4051-a9a8-8435c5d5020b None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:31.421416 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6a2ca-5736-4051-a9a8-8435c5d5020b None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:31.421692 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6a2ca-5736-4051-a9a8-8435c5d5020b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:31.431359 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6a2ca-5736-4051-a9a8-8435c5d5020b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:31.433578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6a2ca-5736-4051-a9a8-8435c5d5020b None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:31.447758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-a4d6a2ca-5736-4051-a9a8-8435c5d5020b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:31.447758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:31.447758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:31.447758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:31.447758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:31.447758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:31.447758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:31.447758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:31.447758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:31.447758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:31.447758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:31.447758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:31.447758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:43:31.447758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:43:31.447758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:31.449386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:31.450509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:43:31.450509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:43:31.450509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:31.450509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:43:31.450509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 620/1265] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:31.455028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ad09d-9a24-4d47-89f2-6bc15d98d6cb None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:31.532701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:31.551094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ad09d-9a24-4d47-89f2-6bc15d98d6cb None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:31.588207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42ec6e-ca70-4afa-9962-fa3f00ebd887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:31.588959 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42ec6e-ca70-4afa-9962-fa3f00ebd887 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:31.589553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42ec6e-ca70-4afa-9962-fa3f00ebd887 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:31.589866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42ec6e-ca70-4afa-9962-fa3f00ebd887 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:31.590792 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42ec6e-ca70-4afa-9962-fa3f00ebd887 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:31.591272 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42ec6e-ca70-4afa-9962-fa3f00ebd887 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:31.593271 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42ec6e-ca70-4afa-9962-fa3f00ebd887 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:31.609865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 646/1266] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:31.616431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6a2ca-5736-4051-a9a8-8435c5d5020b None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:31.714489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6a2ca-5736-4051-a9a8-8435c5d5020b None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:31.733244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:31.747847 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcf5d8-b13f-4746-9096-84a1a36c8580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:31.748473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcf5d8-b13f-4746-9096-84a1a36c8580 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:31.748755 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcf5d8-b13f-4746-9096-84a1a36c8580 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:31.749028 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bcf5d8-b13f-4746-9096-84a1a36c8580 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:31.756976 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bcf5d8-b13f-4746-9096-84a1a36c8580 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:31.758409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bcf5d8-b13f-4746-9096-84a1a36c8580 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:31.769990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-d5bcf5d8-b13f-4746-9096-84a1a36c8580 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 12:43:31.769990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:31.769990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:31.769990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:31.769990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:31.769990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:31.769990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:31.769990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:31.769990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:31.769990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:31.769990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:31.769990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:31.769990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:43:31.769990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:43:31.769990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:43:31.769990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:31.771101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:31.772307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:31.772307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:31.772307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:43:31.772307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:43:31.772307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:43:31.772307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:43:31.772307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:43:31.772307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:43:31.772307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 12:43:31.772307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:43:31.772307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 621/1267] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:31.776740 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42ec6e-ca70-4afa-9962-fa3f00ebd887 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:31.873001 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42ec6e-ca70-4afa-9962-fa3f00ebd887 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:31.905604 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5070fd-b1d3-4f94-b708-531a6f8b1701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:31.906215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5070fd-b1d3-4f94-b708-531a6f8b1701 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:31.906514 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5070fd-b1d3-4f94-b708-531a6f8b1701 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:31.906793 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5070fd-b1d3-4f94-b708-531a6f8b1701 None admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:31.907574 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5070fd-b1d3-4f94-b708-531a6f8b1701 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:31.907878 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5070fd-b1d3-4f94-b708-531a6f8b1701 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:31.909281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5070fd-b1d3-4f94-b708-531a6f8b1701 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:31.918059 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:31.926321 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 647/1268] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 3 12:43:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:31.932125 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcf5d8-b13f-4746-9096-84a1a36c8580 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:32.028755 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bcf5d8-b13f-4746-9096-84a1a36c8580 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:32.061769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353d802-8174-48c0-a5f6-fe035fdfc349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:32.062394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353d802-8174-48c0-a5f6-fe035fdfc349 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:32.062702 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353d802-8174-48c0-a5f6-fe035fdfc349 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:32.062978 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353d802-8174-48c0-a5f6-fe035fdfc349 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:32.073087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353d802-8174-48c0-a5f6-fe035fdfc349 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:32.074570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353d802-8174-48c0-a5f6-fe035fdfc349 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:32.088303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-8353d802-8174-48c0-a5f6-fe035fdfc349 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:32.088303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:32.088303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:32.088303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:32.088303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:32.088303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:32.088303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:32.088303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:32.088303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:32.088303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:32.088303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:32.088303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:32.088303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:43:32.088303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:43:32.088303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:32.089402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:32.090369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:43:32.090369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:43:32.090369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:32.090369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:43:32.090369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 622/1269] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:32.094924 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5070fd-b1d3-4f94-b708-531a6f8b1701 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:32.191337 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5070fd-b1d3-4f94-b708-531a6f8b1701 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:32.224036 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0e3e1-e3b8-4071-b269-b8b632ebdf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:32.224642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0e3e1-e3b8-4071-b269-b8b632ebdf85 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:32.224926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0e3e1-e3b8-4071-b269-b8b632ebdf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:32.225204 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0e3e1-e3b8-4071-b269-b8b632ebdf85 None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:32.225889 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0e3e1-e3b8-4071-b269-b8b632ebdf85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:32.226194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0e3e1-e3b8-4071-b269-b8b632ebdf85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:32.227587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0e3e1-e3b8-4071-b269-b8b632ebdf85 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:32.238926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 648/1270] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:32.244402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353d802-8174-48c0-a5f6-fe035fdfc349 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:32.274720 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:32.340706 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353d802-8174-48c0-a5f6-fe035fdfc349 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:32.372681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb560da-4dbe-4739-adc1-aa8be88e7f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:32.373298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb560da-4dbe-4739-adc1-aa8be88e7f04 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:32.373579 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb560da-4dbe-4739-adc1-aa8be88e7f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:32.373851 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb560da-4dbe-4739-adc1-aa8be88e7f04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:32.380235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb560da-4dbe-4739-adc1-aa8be88e7f04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:32.381660 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb560da-4dbe-4739-adc1-aa8be88e7f04 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:32.392682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-9eb560da-4dbe-4739-adc1-aa8be88e7f04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:43:32.392682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:32.392682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:32.392682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:32.392682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:32.392682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:32.392682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:32.392682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:32.392682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:32.392682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:32.392682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:32.392682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:32.392682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:43:32.392682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:43:32.392682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:32.393721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:32.394900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:43:32.394900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:43:32.394900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:43:32.394900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:43:32.394900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:43:32.394900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:43:32.394900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:43:32.394900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:43:32.394900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 623/1271] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:43:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:32.399353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0e3e1-e3b8-4071-b269-b8b632ebdf85 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:32.455211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:32.496137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0e3e1-e3b8-4071-b269-b8b632ebdf85 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:32.527932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309b144-f253-40e4-bce3-141fd9d48b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:32.528536 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309b144-f253-40e4-bce3-141fd9d48b7c None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:32.528814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309b144-f253-40e4-bce3-141fd9d48b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:32.529086 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309b144-f253-40e4-bce3-141fd9d48b7c None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:32.529822 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309b144-f253-40e4-bce3-141fd9d48b7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:32.530128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309b144-f253-40e4-bce3-141fd9d48b7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:32.531561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309b144-f253-40e4-bce3-141fd9d48b7c None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:32.543449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 649/1272] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:43:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:43:32.549268 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb560da-4dbe-4739-adc1-aa8be88e7f04 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:32.649427 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:32.659272 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb560da-4dbe-4739-adc1-aa8be88e7f04 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:32.691369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa31c2-33dc-4fe0-b208-706c9ea68e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:32.692036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa31c2-33dc-4fe0-b208-706c9ea68e91 None admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:32.692319 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa31c2-33dc-4fe0-b208-706c9ea68e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:32.692596 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa31c2-33dc-4fe0-b208-706c9ea68e91 None admin] PATH_INFO: `/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/5094263ac471481db48783b19192e01d/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:32.694540 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa31c2-33dc-4fe0-b208-706c9ea68e91 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0774035ad0f41c986771ef4a41c7d60, user_id=5094263ac471481db48783b19192e01d, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:32.695965 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa31c2-33dc-4fe0-b208-706c9ea68e91 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:32.708422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 624/1273] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 3 12:43:32 2022] PUT /identity/v3/projects/c0774035ad0f41c986771ef4a41c7d60/users/5094263ac471481db48783b19192e01d/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:43:34.666918 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02f4b2-cd35-4878-be68-751eb96139fa None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:34.667482 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02f4b2-cd35-4878-be68-751eb96139fa None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:34.667568 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02f4b2-cd35-4878-be68-751eb96139fa None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:34.669055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 650/1274] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:43:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:43:34.674682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c3eb8-a322-486b-a327-b50ccfa9eaea None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:34.675128 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c3eb8-a322-486b-a327-b50ccfa9eaea None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:34.675259 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c3eb8-a322-486b-a327-b50ccfa9eaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:34.694299 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a94c3eb8-a322-486b-a327-b50ccfa9eaea None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:43:34.695521 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a94c3eb8-a322-486b-a327-b50ccfa9eaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:43:34.760043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 625/1275] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:43:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 03 12:43:34.783418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02f4b2-cd35-4878-be68-751eb96139fa None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:34.883481 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02f4b2-cd35-4878-be68-751eb96139fa None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:34.901829 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:34.918506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b69921-0415-494a-b238-85e0aca5593f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:34.919083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b69921-0415-494a-b238-85e0aca5593f None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:34.919356 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b69921-0415-494a-b238-85e0aca5593f None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:34.919624 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b69921-0415-494a-b238-85e0aca5593f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:34.929202 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b69921-0415-494a-b238-85e0aca5593f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 03 12:43:34.934900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-f3b69921-0415-494a-b238-85e0aca5593f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:43:34.934900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:34.934900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:34.934900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:34.934900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:34.934900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:34.934900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:34.934900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:34.934900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:34.934900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:34.934900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:34.934900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:34.934900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 03 12:43:34.934900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 03 12:43:34.934900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 03 12:43:34.934900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 03 12:43:34.936041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 03 12:43:34.936041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 03 12:43:34.936041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 03 12:43:34.936041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 03 12:43:34.936041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:34.936041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:34.936041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:34.936041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:34.936041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:34.936041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:34.936041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:34.936041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:34.936041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:34.936041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:34.936041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:34.936041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:34.936944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:34.936944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:34.936944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:34.936944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:34.936944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 03 12:43:34.936944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 03 12:43:34.936944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 03 12:43:34.936944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 03 12:43:34.936944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 03 12:43:34.936944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 03 12:43:34.936944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 03 12:43:34.936944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:34.936944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 651/1276] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 3 12:43:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:34.942015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c3eb8-a322-486b-a327-b50ccfa9eaea None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:35.019524 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:35.038183 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c3eb8-a322-486b-a327-b50ccfa9eaea None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:35.073967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f9110-4efc-4fea-adc3-fda4189ff424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:35.074611 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f9110-4efc-4fea-adc3-fda4189ff424 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:35.074893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f9110-4efc-4fea-adc3-fda4189ff424 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:35.075165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f9110-4efc-4fea-adc3-fda4189ff424 None admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:35.075871 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f9110-4efc-4fea-adc3-fda4189ff424 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:35.076175 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f9110-4efc-4fea-adc3-fda4189ff424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:35.077525 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f9110-4efc-4fea-adc3-fda4189ff424 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:35.085323 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 626/1277] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 3 12:43:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:35.090990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b69921-0415-494a-b238-85e0aca5593f None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:35.187791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b69921-0415-494a-b238-85e0aca5593f None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:35.219770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ac887-e8ba-4e1a-8a3f-85c64cd0a6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:35.220388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ac887-e8ba-4e1a-8a3f-85c64cd0a6cc None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:35.220668 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ac887-e8ba-4e1a-8a3f-85c64cd0a6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:35.220937 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ac887-e8ba-4e1a-8a3f-85c64cd0a6cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:35.231034 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ac887-e8ba-4e1a-8a3f-85c64cd0a6cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:35.232463 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7ac887-e8ba-4e1a-8a3f-85c64cd0a6cc None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:35.246512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-ed7ac887-e8ba-4e1a-8a3f-85c64cd0a6cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:35.246512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:35.246512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:35.246512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:35.246512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:35.246512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:35.246512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:35.246512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:35.246512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:35.246512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:35.246512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:35.246512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:35.246512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:43:35.246512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:43:35.246512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:35.247582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:35.248884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:43:35.248884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:43:35.248884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:35.248884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:35.248884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 652/1278] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:35.253181 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f9110-4efc-4fea-adc3-fda4189ff424 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:35.350085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f9110-4efc-4fea-adc3-fda4189ff424 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:35.382347 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd3646-07c1-4dbb-80c8-f7ee832a7bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:35.382976 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd3646-07c1-4dbb-80c8-f7ee832a7bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:35.383252 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd3646-07c1-4dbb-80c8-f7ee832a7bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:35.383520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd3646-07c1-4dbb-80c8-f7ee832a7bc1 None admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:35.384196 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd3646-07c1-4dbb-80c8-f7ee832a7bc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:35.384496 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd3646-07c1-4dbb-80c8-f7ee832a7bc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:35.385831 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd3646-07c1-4dbb-80c8-f7ee832a7bc1 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:35.397563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 627/1279] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:35.402802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ac887-e8ba-4e1a-8a3f-85c64cd0a6cc None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:35.432079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:35.499037 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ac887-e8ba-4e1a-8a3f-85c64cd0a6cc None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:35.530749 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae7ebc-16a2-4b6c-bedf-72c1e102d065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:35.531361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae7ebc-16a2-4b6c-bedf-72c1e102d065 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:35.531636 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae7ebc-16a2-4b6c-bedf-72c1e102d065 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:35.531902 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae7ebc-16a2-4b6c-bedf-72c1e102d065 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:35.538896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae7ebc-16a2-4b6c-bedf-72c1e102d065 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:35.540302 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ae7ebc-16a2-4b6c-bedf-72c1e102d065 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:35.551538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-63ae7ebc-16a2-4b6c-bedf-72c1e102d065 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 03 12:43:35.551538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:35.551538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:35.551538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:35.551538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:35.551538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:35.551538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:35.551538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:35.551538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:35.551538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:35.551538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:35.551538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:35.551538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 03 12:43:35.551538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 03 12:43:35.551538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 03 12:43:35.551538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:35.552661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:35.554113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:35.554113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:35.554113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 03 12:43:35.554113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 03 12:43:35.554113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 03 12:43:35.554113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 03 12:43:35.554113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 03 12:43:35.554113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 03 12:43:35.554113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 03 12:43:35.554113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:35.554113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 653/1280] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:35.559118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd3646-07c1-4dbb-80c8-f7ee832a7bc1 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:35.562822 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:35.662125 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd3646-07c1-4dbb-80c8-f7ee832a7bc1 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:35.695253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7ef8f-7522-430f-b3c3-44e981e50d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:35.695859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7ef8f-7522-430f-b3c3-44e981e50d20 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:35.696135 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7ef8f-7522-430f-b3c3-44e981e50d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:35.696405 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7ef8f-7522-430f-b3c3-44e981e50d20 None admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:35.697182 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7ef8f-7522-430f-b3c3-44e981e50d20 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:35.697484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7ef8f-7522-430f-b3c3-44e981e50d20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:35.698976 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7ef8f-7522-430f-b3c3-44e981e50d20 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:35.715132 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 628/1281] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 3 12:43:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:35.720832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae7ebc-16a2-4b6c-bedf-72c1e102d065 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:35.807713 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:35.818300 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ae7ebc-16a2-4b6c-bedf-72c1e102d065 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:35.849861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f0e65-3b46-4d88-9bec-60a675c85d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:35.850464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f0e65-3b46-4d88-9bec-60a675c85d1a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:35.850772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f0e65-3b46-4d88-9bec-60a675c85d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:35.851043 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f0e65-3b46-4d88-9bec-60a675c85d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:35.915854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f0e65-3b46-4d88-9bec-60a675c85d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:35.917795 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f0e65-3b46-4d88-9bec-60a675c85d1a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:36.002182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:36.012827 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 654/1282] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 3 12:43:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 03 12:43:36.018555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7ef8f-7522-430f-b3c3-44e981e50d20 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:36.077870 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:36.116258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7ef8f-7522-430f-b3c3-44e981e50d20 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:36.147996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609bef1-2f51-4bf4-8346-bacdb11114d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:36.148614 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609bef1-2f51-4bf4-8346-bacdb11114d9 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:36.148890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609bef1-2f51-4bf4-8346-bacdb11114d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:36.149158 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609bef1-2f51-4bf4-8346-bacdb11114d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:36.159194 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609bef1-2f51-4bf4-8346-bacdb11114d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:36.160612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609bef1-2f51-4bf4-8346-bacdb11114d9 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:36.175115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-8609bef1-2f51-4bf4-8346-bacdb11114d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:36.175115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:36.175115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:36.175115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:36.175115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:36.175115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:36.175115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:36.175115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:36.175115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:36.175115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:36.175115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:36.175115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:36.175115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 03 12:43:36.175115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 03 12:43:36.175115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:36.176191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:36.177165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 03 12:43:36.177165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 03 12:43:36.177165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 03 12:43:36.177165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:43:36.177165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 629/1283] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:36.181999 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f0e65-3b46-4d88-9bec-60a675c85d1a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:36.277376 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f0e65-3b46-4d88-9bec-60a675c85d1a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:36.310250 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43290778-84c3-4779-8aee-3494986dd21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:36.310880 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43290778-84c3-4779-8aee-3494986dd21a None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:36.311162 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43290778-84c3-4779-8aee-3494986dd21a None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:36.311439 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43290778-84c3-4779-8aee-3494986dd21a None admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:36.312119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43290778-84c3-4779-8aee-3494986dd21a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:36.312422 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43290778-84c3-4779-8aee-3494986dd21a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:36.313766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43290778-84c3-4779-8aee-3494986dd21a None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:36.324973 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 655/1284] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 3 12:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:36.330246 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609bef1-2f51-4bf4-8346-bacdb11114d9 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:36.426634 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609bef1-2f51-4bf4-8346-bacdb11114d9 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:36.439919 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:36.459946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc8988-9796-43f9-9bf1-d881beabb079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:36.460575 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc8988-9796-43f9-9bf1-d881beabb079 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:36.460874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc8988-9796-43f9-9bf1-d881beabb079 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:36.461149 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc8988-9796-43f9-9bf1-d881beabb079 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:36.468184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc8988-9796-43f9-9bf1-d881beabb079 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:36.469608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc8988-9796-43f9-9bf1-d881beabb079 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:36.481508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application [None req-00bc8988-9796-43f9-9bf1-d881beabb079 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:43:36.481508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:36.481508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:36.481508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:36.481508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:36.481508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:36.481508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:36.481508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:36.481508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:36.481508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:36.481508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:36.481508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:36.481508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 03 12:43:36.481508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 03 12:43:36.481508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:36.482391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:36.483398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 03 12:43:36.483398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 03 12:43:36.483398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 03 12:43:36.483398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 03 12:43:36.483398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 03 12:43:36.483398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 03 12:43:36.483398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 03 12:43:36.483398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: ERROR keystone.server.flask.application Aug 03 12:43:36.483398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 630/1285] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 3 12:43:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:36.488232 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43290778-84c3-4779-8aee-3494986dd21a None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:36.536207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:36.588098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43290778-84c3-4779-8aee-3494986dd21a None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:36.621193 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c308b26-fa67-4e4d-8b72-bfef1aee00a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:36.621803 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308b26-fa67-4e4d-8b72-bfef1aee00a5 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:36.622085 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308b26-fa67-4e4d-8b72-bfef1aee00a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:36.622359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308b26-fa67-4e4d-8b72-bfef1aee00a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:36.623137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c308b26-fa67-4e4d-8b72-bfef1aee00a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:36.623445 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c308b26-fa67-4e4d-8b72-bfef1aee00a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:36.624855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c308b26-fa67-4e4d-8b72-bfef1aee00a5 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:36.636851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 656/1286] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 3 12:43:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:36.642893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc8988-9796-43f9-9bf1-d881beabb079 None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:36.739317 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc8988-9796-43f9-9bf1-d881beabb079 None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:36.778350 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67382fcd-0e2e-44ab-bdf7-90482ca76146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:36.778984 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67382fcd-0e2e-44ab-bdf7-90482ca76146 None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:36.779268 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67382fcd-0e2e-44ab-bdf7-90482ca76146 None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:36.779544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67382fcd-0e2e-44ab-bdf7-90482ca76146 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:36.780323 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67382fcd-0e2e-44ab-bdf7-90482ca76146 None admin] RBAC: Adding query filter params (role.id=74ec741f50da4dd1b83c16cb6ecd8446, scope.project.id=c0774035ad0f41c986771ef4a41c7d60, user.id=5094263ac471481db48783b19192e01d) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:36.780633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67382fcd-0e2e-44ab-bdf7-90482ca76146 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:36.782046 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67382fcd-0e2e-44ab-bdf7-90482ca76146 None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:36.787733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 631/1287] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 3 12:43:36 2022] GET /identity/v3/role_assignments?user.id=5094263ac471481db48783b19192e01d&scope.project.id=c0774035ad0f41c986771ef4a41c7d60&role.id=74ec741f50da4dd1b83c16cb6ecd8446 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:38.784433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48521691-5c02-427d-a1a9-d65f4773559f None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:38.784974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48521691-5c02-427d-a1a9-d65f4773559f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:38.784974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48521691-5c02-427d-a1a9-d65f4773559f None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:38.786304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 657/1288] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:43:38.791300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7228be-c2f2-45b5-85a2-113307de0f98 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:38.791761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7228be-c2f2-45b5-85a2-113307de0f98 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:38.791857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7228be-c2f2-45b5-85a2-113307de0f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:38.811037 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-ba7228be-c2f2-45b5-85a2-113307de0f98 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:43:38.812238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ba7228be-c2f2-45b5-85a2-113307de0f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:43:38.840100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:38.877645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 632/1289] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:43:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:43:38.899315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48521691-5c02-427d-a1a9-d65f4773559f None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:39.003304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48521691-5c02-427d-a1a9-d65f4773559f None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:39.013358 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:39.035985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3be72-009e-41d3-a8a9-a44dfdd39f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:39.036549 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3be72-009e-41d3-a8a9-a44dfdd39f26 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:39.036824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3be72-009e-41d3-a8a9-a44dfdd39f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:39.037091 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3be72-009e-41d3-a8a9-a44dfdd39f26 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:39.037703 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3be72-009e-41d3-a8a9-a44dfdd39f26 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:39.039069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3be72-009e-41d3-a8a9-a44dfdd39f26 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-49d3be72-009e-41d3-a8a9-a44dfdd39f26 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:39.048134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:39.049342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:39.049342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:39.049342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:39.049342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:39.049342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:39.049342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:39.049342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:43:39.049342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:43:39.049342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:43:39.049342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:43:39.049342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:39.049342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:39.049342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:39.049342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:39.049342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:39.049342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:39.050307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:39.050307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:39.050307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:39.050307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:39.050307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:39.050307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:39.050307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:39.050307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:39.050307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:39.050307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:39.050307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:43:39.050307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:43:39.050307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 12:43:39.050307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:39.050307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 658/1290] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 3 12:43:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:39.055259 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7228be-c2f2-45b5-85a2-113307de0f98 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:39.151076 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7228be-c2f2-45b5-85a2-113307de0f98 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:39.184778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df60a22-85f1-4cde-85e1-f5abdf4e5e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:39.185384 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df60a22-85f1-4cde-85e1-f5abdf4e5e4e None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:39.185666 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df60a22-85f1-4cde-85e1-f5abdf4e5e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:39.185942 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df60a22-85f1-4cde-85e1-f5abdf4e5e4e None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:39.186641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df60a22-85f1-4cde-85e1-f5abdf4e5e4e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:39.186947 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df60a22-85f1-4cde-85e1-f5abdf4e5e4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:39.188315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df60a22-85f1-4cde-85e1-f5abdf4e5e4e None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:39.192985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 633/1291] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Aug 3 12:43:39 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:39.198676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3be72-009e-41d3-a8a9-a44dfdd39f26 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:39.293175 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3be72-009e-41d3-a8a9-a44dfdd39f26 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:39.324822 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8195b517-53ae-42da-9030-11526420f3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:39.325410 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195b517-53ae-42da-9030-11526420f3cb None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:39.325686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195b517-53ae-42da-9030-11526420f3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:39.325952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195b517-53ae-42da-9030-11526420f3cb None admin] PATH_INFO: `/v3/services` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:39.326635 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195b517-53ae-42da-9030-11526420f3cb None admin] RBAC: Adding query filter params (type=placement) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:39.326934 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195b517-53ae-42da-9030-11526420f3cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:39.328277 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8195b517-53ae-42da-9030-11526420f3cb None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:39.332472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 659/1292] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Aug 3 12:43:39 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:41.315891 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5802ac6-3c68-4cbd-bba5-4663ee4800ed None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:41.316438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5802ac6-3c68-4cbd-bba5-4663ee4800ed None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:41.316438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5802ac6-3c68-4cbd-bba5-4663ee4800ed None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:41.317799 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 634/1293] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:43:41.322799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79298d81-5316-45fd-8b67-13a528e1ea8b None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:41.323214 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79298d81-5316-45fd-8b67-13a528e1ea8b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:41.323890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79298d81-5316-45fd-8b67-13a528e1ea8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:41.341448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-79298d81-5316-45fd-8b67-13a528e1ea8b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:43:41.342663 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-79298d81-5316-45fd-8b67-13a528e1ea8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:43:41.371116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:41.407153 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 660/1294] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:43:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:43:41.428584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5802ac6-3c68-4cbd-bba5-4663ee4800ed None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:41.459819 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:41.528232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5802ac6-3c68-4cbd-bba5-4663ee4800ed None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:41.561590 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550f0a2-dc96-44ea-8d00-84d0e86e98aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:41.562197 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550f0a2-dc96-44ea-8d00-84d0e86e98aa None admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:41.562497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550f0a2-dc96-44ea-8d00-84d0e86e98aa None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:41.562789 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550f0a2-dc96-44ea-8d00-84d0e86e98aa None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:41.563490 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550f0a2-dc96-44ea-8d00-84d0e86e98aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:41.564849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550f0a2-dc96-44ea-8d00-84d0e86e98aa None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:41.582665 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 635/1295] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 3 12:43:41 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:43:43.567504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c525a6-31ae-4a7c-ae25-2dcd2ff799b8 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:43.568050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c525a6-31ae-4a7c-ae25-2dcd2ff799b8 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:43.568199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c525a6-31ae-4a7c-ae25-2dcd2ff799b8 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:43.569476 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 661/1296] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:43:43.574646 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecede39-654f-47d6-bf3a-a199550e1b5a None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:43.575026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecede39-654f-47d6-bf3a-a199550e1b5a None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:43.575210 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecede39-654f-47d6-bf3a-a199550e1b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:43.593335 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-1ecede39-654f-47d6-bf3a-a199550e1b5a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:43:43.594556 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1ecede39-654f-47d6-bf3a-a199550e1b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:43:43.664880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 636/1297] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:43:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:43:43.687390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c525a6-31ae-4a7c-ae25-2dcd2ff799b8 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:43.783942 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c525a6-31ae-4a7c-ae25-2dcd2ff799b8 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:43.817178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c90fee-8e90-422c-804b-b8a11d2c1a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:43.817857 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c90fee-8e90-422c-804b-b8a11d2c1a75 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:43.818086 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c90fee-8e90-422c-804b-b8a11d2c1a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:43.818360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c90fee-8e90-422c-804b-b8a11d2c1a75 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:43.819003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c90fee-8e90-422c-804b-b8a11d2c1a75 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:43.820348 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c90fee-8e90-422c-804b-b8a11d2c1a75 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-82c90fee-8e90-422c-804b-b8a11d2c1a75 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:43.829456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:43.830645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:43.830645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:43.830645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:43.830645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:43.830645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:43.830645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:43.830645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:43:43.830645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:43:43.830645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:43:43.830645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:43:43.830645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:43.830645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:43.830645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:43.830645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:43.830645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:43.830645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:43.831521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:43.831521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:43.831521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:43.831521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:43.831521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:43.831521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:43.831521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:43.831521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:43.831521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:43.831521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:43.831521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:43:43.831521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:43:43.831521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 12:43:43.831521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:43.831521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 662/1298] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 3 12:43:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:43.836500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecede39-654f-47d6-bf3a-a199550e1b5a None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:43.856269 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:43.935634 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecede39-654f-47d6-bf3a-a199550e1b5a None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:43.968942 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829d123-fe92-4145-902c-56e5008aaabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:43.969547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829d123-fe92-4145-902c-56e5008aaabf None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:43.969822 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829d123-fe92-4145-902c-56e5008aaabf None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:43.970090 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7829d123-fe92-4145-902c-56e5008aaabf None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:43.970807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829d123-fe92-4145-902c-56e5008aaabf None admin] RBAC: Adding query filter params (name=placement) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:43.971105 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829d123-fe92-4145-902c-56e5008aaabf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:43.972460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7829d123-fe92-4145-902c-56e5008aaabf None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:43.978457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 637/1299] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Aug 3 12:43:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:43.984387 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c90fee-8e90-422c-804b-b8a11d2c1a75 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:44.014780 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:44.080773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c90fee-8e90-422c-804b-b8a11d2c1a75 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:44.117661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fa6a49-119b-43e2-a17c-78b5ed478df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:44.118269 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa6a49-119b-43e2-a17c-78b5ed478df5 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:44.118580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa6a49-119b-43e2-a17c-78b5ed478df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:44.118859 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa6a49-119b-43e2-a17c-78b5ed478df5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:44.119607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa6a49-119b-43e2-a17c-78b5ed478df5 None admin] RBAC: Adding query filter params (interface=public, service_id=38971aa9ca79425c96ebfbbaf38b7943, region_id=RegionOne) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:44.119919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa6a49-119b-43e2-a17c-78b5ed478df5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:44.121279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fa6a49-119b-43e2-a17c-78b5ed478df5 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:44.126190 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 663/1300] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 3 12:43:43 2022] GET /identity/v3/endpoints?service_id=38971aa9ca79425c96ebfbbaf38b7943&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:44.131800 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829d123-fe92-4145-902c-56e5008aaabf None admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:44.210129 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:44.227840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7829d123-fe92-4145-902c-56e5008aaabf None admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:44.260245 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e8098-c0f2-4807-a244-fe0a5bedd2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:44.261476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e8098-c0f2-4807-a244-fe0a5bedd2ae None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:44.261758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e8098-c0f2-4807-a244-fe0a5bedd2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:44.262031 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e8098-c0f2-4807-a244-fe0a5bedd2ae None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:44.262703 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e8098-c0f2-4807-a244-fe0a5bedd2ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:44.264050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e8098-c0f2-4807-a244-fe0a5bedd2ae None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:44.270653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 638/1301] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 3 12:43:44 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:43:46.262875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571d33c-fdcd-4863-9238-de6501739a45 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:46.263410 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571d33c-fdcd-4863-9238-de6501739a45 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:46.263523 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571d33c-fdcd-4863-9238-de6501739a45 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:46.264777 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 664/1302] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:43:46.269916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a24ed-bb97-45df-aa5c-fed784100d3e None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:46.270309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a24ed-bb97-45df-aa5c-fed784100d3e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:46.270458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a24ed-bb97-45df-aa5c-fed784100d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:46.287925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-031a24ed-bb97-45df-aa5c-fed784100d3e None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:43:46.289093 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-031a24ed-bb97-45df-aa5c-fed784100d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:43:46.352861 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 639/1303] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:43:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 03 12:43:46.375644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571d33c-fdcd-4863-9238-de6501739a45 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:46.466425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:46.473350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571d33c-fdcd-4863-9238-de6501739a45 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:46.505762 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5fa23-f925-4868-93b0-3ef758f97be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:46.506383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5fa23-f925-4868-93b0-3ef758f97be0 None admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:46.506686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5fa23-f925-4868-93b0-3ef758f97be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:46.506958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a5fa23-f925-4868-93b0-3ef758f97be0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:46.507578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5fa23-f925-4868-93b0-3ef758f97be0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:46.508908 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a5fa23-f925-4868-93b0-3ef758f97be0 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application [None req-19a5fa23-f925-4868-93b0-3ef758f97be0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 03 12:43:46.517491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 03 12:43:46.518800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 03 12:43:46.518800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 03 12:43:46.518800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 03 12:43:46.518800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 03 12:43:46.518800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 03 12:43:46.518800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 03 12:43:46.518800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 03 12:43:46.518800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 03 12:43:46.518800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 03 12:43:46.518800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 03 12:43:46.518800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 03 12:43:46.518800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 03 12:43:46.518800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 03 12:43:46.518800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 03 12:43:46.518800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 03 12:43:46.518800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.get_or_create( Aug 03 12:43:46.520032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 03 12:43:46.520032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application with Lock( Aug 03 12:43:46.520032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 03 12:43:46.520032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self._enter() Aug 03 12:43:46.520032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 03 12:43:46.520032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 03 12:43:46.520032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 03 12:43:46.520032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application return self.creator() Aug 03 12:43:46.520032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 03 12:43:46.520032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application created_value = creator( Aug 03 12:43:46.520032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 03 12:43:46.520032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 03 12:43:46.520032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 03 12:43:46.520032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: ERROR keystone.server.flask.application Aug 03 12:43:46.520032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 665/1304] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 3 12:43:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 03 12:43:46.524511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a24ed-bb97-45df-aa5c-fed784100d3e None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:46.613016 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:46.640368 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a24ed-bb97-45df-aa5c-fed784100d3e None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:46.674156 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14a674f-7210-4de4-9d1d-9a7edb58bfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:46.674796 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a674f-7210-4de4-9d1d-9a7edb58bfcd None admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:46.675078 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a674f-7210-4de4-9d1d-9a7edb58bfcd None admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:46.675349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a674f-7210-4de4-9d1d-9a7edb58bfcd None admin] PATH_INFO: `/v3/services` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:46.676025 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a674f-7210-4de4-9d1d-9a7edb58bfcd None admin] RBAC: Adding query filter params (name=placement) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:43:46.676329 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a674f-7210-4de4-9d1d-9a7edb58bfcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:46.677685 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14a674f-7210-4de4-9d1d-9a7edb58bfcd None admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:46.683020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 640/1305] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Aug 3 12:43:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:43:46.688850 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5fa23-f925-4868-93b0-3ef758f97be0 None admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:43:46.783718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a5fa23-f925-4868-93b0-3ef758f97be0 None admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:43:46.815858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0c500-8665-49ac-a8be-196238daa934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:43:46.816458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0c500-8665-49ac-a8be-196238daa934 None admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:43:46.816733 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0c500-8665-49ac-a8be-196238daa934 None admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:43:46.817001 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0c500-8665-49ac-a8be-196238daa934 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:43:46.817688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0c500-8665-49ac-a8be-196238daa934 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:43:46.819056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0c500-8665-49ac-a8be-196238daa934 None admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:43:46.837993 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 666/1306] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:43:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:43:56.680094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=30 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:43:56.834031 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:44:49.152014 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b7e1a-bb17-46b2-ae42-8776efe188bc None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:44:49.152513 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b7e1a-bb17-46b2-ae42-8776efe188bc None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:44:49.152607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720b7e1a-bb17-46b2-ae42-8776efe188bc None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:44:49.159064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fcece-0a91-4379-b6b4-5dfe290af458 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:44:49.159506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fcece-0a91-4379-b6b4-5dfe290af458 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:44:49.159599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662fcece-0a91-4379-b6b4-5dfe290af458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:44:49.164339 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 641/1307] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:44:49 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:44:49.333444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-662fcece-0a91-4379-b6b4-5dfe290af458 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:44:49.336332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-662fcece-0a91-4379-b6b4-5dfe290af458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:44:49.494518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 667/1308] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:44:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:44:49.537245 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9951b-3027-405e-a572-5b38764efdf2 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:44:49.537705 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9951b-3027-405e-a572-5b38764efdf2 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:44:49.538111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9951b-3027-405e-a572-5b38764efdf2 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:44:49.540473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 642/1309] 10.222.0.65 () {54 vars in 965 bytes} [Wed Aug 3 12:44:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:44:49.547260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9170b4-3f9f-4671-8819-97ef9f9e8163 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:44:49.547554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9170b4-3f9f-4671-8819-97ef9f9e8163 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:44:49.547821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9170b4-3f9f-4671-8819-97ef9f9e8163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:44:49.643084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-ae9170b4-3f9f-4671-8819-97ef9f9e8163 None None] MFA Rules not processed for user `52ff458fd7f74b99a745edbcf13a114b`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:44:49.644727 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ae9170b4-3f9f-4671-8819-97ef9f9e8163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:44:49.669078 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:44:49.728627 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 668/1310] 10.222.0.65 () {58 vars in 1062 bytes} [Wed Aug 3 12:44:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:44:49.737767 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9951b-3027-405e-a572-5b38764efdf2 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:44:49.739543 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-84a9951b-3027-405e-a572-5b38764efdf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:44:49.851230 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9951b-3027-405e-a572-5b38764efdf2 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:44:49.928716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d2d23-7480-4a1b-9640-91f72cc04418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:44:49.929254 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d2d23-7480-4a1b-9640-91f72cc04418 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:44:49.929533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d2d23-7480-4a1b-9640-91f72cc04418 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:44:49.929800 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d2d23-7480-4a1b-9640-91f72cc04418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:44:49.932263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-953d2d23-7480-4a1b-9640-91f72cc04418 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:44:49.988682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:44:50.048511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d2d23-7480-4a1b-9640-91f72cc04418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:44:50.050127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d2d23-7480-4a1b-9640-91f72cc04418 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:44:50.070721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 643/1311] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:44:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:44:52.387083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44681cca-189d-4478-bc61-a07b6f418b9e None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:44:52.387654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44681cca-189d-4478-bc61-a07b6f418b9e None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:44:52.387654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44681cca-189d-4478-bc61-a07b6f418b9e None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:44:52.389074 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 669/1312] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:44:52.394101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677a034-def1-4307-94c9-86bcfb4e700d None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:44:52.394511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677a034-def1-4307-94c9-86bcfb4e700d None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:44:52.394679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677a034-def1-4307-94c9-86bcfb4e700d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:44:52.467515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-4677a034-def1-4307-94c9-86bcfb4e700d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:44:52.468733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-4677a034-def1-4307-94c9-86bcfb4e700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:44:52.572435 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 644/1313] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:44:52.610268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44681cca-189d-4478-bc61-a07b6f418b9e None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:44:52.612002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-44681cca-189d-4478-bc61-a07b6f418b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:44:52.720640 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44681cca-189d-4478-bc61-a07b6f418b9e None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:44:52.793256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:44:52.796098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84826861-3d84-49d9-a6b7-d5effd3d8e66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:44:52.796756 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84826861-3d84-49d9-a6b7-d5effd3d8e66 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:44:52.797033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84826861-3d84-49d9-a6b7-d5effd3d8e66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:44:52.797347 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84826861-3d84-49d9-a6b7-d5effd3d8e66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:44:52.799677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-84826861-3d84-49d9-a6b7-d5effd3d8e66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:44:52.916454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84826861-3d84-49d9-a6b7-d5effd3d8e66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:44:52.917909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84826861-3d84-49d9-a6b7-d5effd3d8e66 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:44:52.938431 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 670/1314] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:44:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:44:55.240794 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2b7a9-4040-49e6-9118-0b7ce2f1b9c1 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:44:55.241383 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2b7a9-4040-49e6-9118-0b7ce2f1b9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:44:55.241383 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2b7a9-4040-49e6-9118-0b7ce2f1b9c1 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:44:55.242756 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 645/1315] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:44:55.248065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae31b0-70c4-4508-8919-8d3ce29fdc47 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:44:55.248471 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae31b0-70c4-4508-8919-8d3ce29fdc47 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:44:55.248627 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae31b0-70c4-4508-8919-8d3ce29fdc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:44:55.314617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-cdae31b0-70c4-4508-8919-8d3ce29fdc47 None None] MFA Rules not processed for user `75e20284884f42e58636018d16cc77a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:44:55.315837 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-cdae31b0-70c4-4508-8919-8d3ce29fdc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:44:55.368359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:44:55.413877 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 671/1316] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:44:55 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:44:55.453754 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2b7a9-4040-49e6-9118-0b7ce2f1b9c1 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:44:55.455321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-7df2b7a9-4040-49e6-9118-0b7ce2f1b9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:44:55.465581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:44:55.558061 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2b7a9-4040-49e6-9118-0b7ce2f1b9c1 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:44:55.630032 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8288f-37f5-4562-b54d-4449e03d5ec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:44:55.630666 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8288f-37f5-4562-b54d-4449e03d5ec0 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:44:55.630945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8288f-37f5-4562-b54d-4449e03d5ec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:44:55.631213 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8288f-37f5-4562-b54d-4449e03d5ec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:44:55.632820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6cf8288f-37f5-4562-b54d-4449e03d5ec0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:44:55.694249 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:44:55.739054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8288f-37f5-4562-b54d-4449e03d5ec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:44:55.740476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8288f-37f5-4562-b54d-4449e03d5ec0 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:44:55.758853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 646/1317] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:44:55 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:44:58.291668 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69b8f8-6ec4-4245-9587-e547eb88ed61 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:44:58.292219 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69b8f8-6ec4-4245-9587-e547eb88ed61 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:44:58.292219 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69b8f8-6ec4-4245-9587-e547eb88ed61 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:44:58.293578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 672/1318] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:44:58.298758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee4f86-991f-42bd-a1b5-143067b2f5b0 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:44:58.299174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee4f86-991f-42bd-a1b5-143067b2f5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:44:58.299310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee4f86-991f-42bd-a1b5-143067b2f5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:44:58.363347 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-ebee4f86-991f-42bd-a1b5-143067b2f5b0 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:44:58.364551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ebee4f86-991f-42bd-a1b5-143067b2f5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:44:58.458087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 647/1319] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:44:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:44:58.497440 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69b8f8-6ec4-4245-9587-e547eb88ed61 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:44:58.499134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-7f69b8f8-6ec4-4245-9587-e547eb88ed61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:44:58.600891 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69b8f8-6ec4-4245-9587-e547eb88ed61 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:44:58.671783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1957b61-ac3e-43ce-94ab-e0918f4f4369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:44:58.672397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1957b61-ac3e-43ce-94ab-e0918f4f4369 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:44:58.672677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1957b61-ac3e-43ce-94ab-e0918f4f4369 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:44:58.672949 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1957b61-ac3e-43ce-94ab-e0918f4f4369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:44:58.674588 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e1957b61-ac3e-43ce-94ab-e0918f4f4369 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:44:58.682549 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:44:58.779936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1957b61-ac3e-43ce-94ab-e0918f4f4369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:44:58.781335 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1957b61-ac3e-43ce-94ab-e0918f4f4369 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:44:58.799983 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 673/1320] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:44:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:45:01.417721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede9732-924b-4c44-affe-2c07b7b53490 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:01.418288 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede9732-924b-4c44-affe-2c07b7b53490 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:01.418288 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede9732-924b-4c44-affe-2c07b7b53490 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:01.419717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 648/1321] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:01.424990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a2d01-d2b5-4dd6-99d8-5706aa45f653 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:01.425394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a2d01-d2b5-4dd6-99d8-5706aa45f653 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:01.425616 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a2d01-d2b5-4dd6-99d8-5706aa45f653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:01.491743 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-d45a2d01-d2b5-4dd6-99d8-5706aa45f653 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:01.493450 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d45a2d01-d2b5-4dd6-99d8-5706aa45f653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:01.550007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:01.601295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 674/1322] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:01.638763 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede9732-924b-4c44-affe-2c07b7b53490 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:01.640424 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-dede9732-924b-4c44-affe-2c07b7b53490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:01.660826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:01.750023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dede9732-924b-4c44-affe-2c07b7b53490 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:45:01.825379 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0d048-56c0-4e1f-aa06-54d33cb69909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:45:01.826001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0d048-56c0-4e1f-aa06-54d33cb69909 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:01.826279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0d048-56c0-4e1f-aa06-54d33cb69909 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:01.826580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0d048-56c0-4e1f-aa06-54d33cb69909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:01.828554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-87e0d048-56c0-4e1f-aa06-54d33cb69909 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:01.900260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:01.938955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0d048-56c0-4e1f-aa06-54d33cb69909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:45:01.940398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0d048-56c0-4e1f-aa06-54d33cb69909 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:45:01.959632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 649/1323] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:45:04.001542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2d77c-68a3-449d-9c51-b15758a76c98 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:04.002073 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2d77c-68a3-449d-9c51-b15758a76c98 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:04.002073 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2d77c-68a3-449d-9c51-b15758a76c98 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:04.003656 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 675/1324] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:04.008528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab49d0-432a-49c7-b248-83e82bd4f0a4 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:04.008953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab49d0-432a-49c7-b248-83e82bd4f0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:04.009083 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab49d0-432a-49c7-b248-83e82bd4f0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:04.073405 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-00ab49d0-432a-49c7-b248-83e82bd4f0a4 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:04.074534 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-00ab49d0-432a-49c7-b248-83e82bd4f0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:04.170012 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 650/1325] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:04.206120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2d77c-68a3-449d-9c51-b15758a76c98 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:04.207679 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-aef2d77c-68a3-449d-9c51-b15758a76c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:04.308654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2d77c-68a3-449d-9c51-b15758a76c98 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:45:04.380669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e680df-d08e-4196-b1cf-a21181d041f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:45:04.381288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e680df-d08e-4196-b1cf-a21181d041f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:04.381567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e680df-d08e-4196-b1cf-a21181d041f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:04.381839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e680df-d08e-4196-b1cf-a21181d041f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:04.383996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e1e680df-d08e-4196-b1cf-a21181d041f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:04.401210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:04.498932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e680df-d08e-4196-b1cf-a21181d041f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:45:04.500352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e680df-d08e-4196-b1cf-a21181d041f7 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:45:04.518973 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 676/1326] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:45:06.566922 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51971587-1a9e-421f-8f9a-3ddab7eb979d None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:06.567442 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51971587-1a9e-421f-8f9a-3ddab7eb979d None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:06.567549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51971587-1a9e-421f-8f9a-3ddab7eb979d None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:06.568891 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 651/1327] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:06.574226 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365398e5-3c11-4515-92ca-4115307f28d2 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:06.574660 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365398e5-3c11-4515-92ca-4115307f28d2 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:06.574816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365398e5-3c11-4515-92ca-4115307f28d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:06.640449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-365398e5-3c11-4515-92ca-4115307f28d2 None None] MFA Rules not processed for user `75e20284884f42e58636018d16cc77a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:06.641644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-365398e5-3c11-4515-92ca-4115307f28d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:06.704791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:06.738511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 677/1328] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:06 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:06.791845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b2e34-bc67-48f1-8198-c7cf24cd2858 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:06.792131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b2e34-bc67-48f1-8198-c7cf24cd2858 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:06.792403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b2e34-bc67-48f1-8198-c7cf24cd2858 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:06.793749 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 652/1329] 10.222.0.65 () {54 vars in 965 bytes} [Wed Aug 3 12:45:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:45:06.799978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b768373-02a3-4d20-95b2-dabbe329bd9a None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:06.800345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b768373-02a3-4d20-95b2-dabbe329bd9a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:06.800685 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b768373-02a3-4d20-95b2-dabbe329bd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:06.881680 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-7b768373-02a3-4d20-95b2-dabbe329bd9a None None] MFA Rules not processed for user `52ff458fd7f74b99a745edbcf13a114b`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:06.882913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-7b768373-02a3-4d20-95b2-dabbe329bd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:06.957481 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 678/1330] 10.222.0.65 () {58 vars in 1062 bytes} [Wed Aug 3 12:45:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:06.966973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b2e34-bc67-48f1-8198-c7cf24cd2858 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:06.968903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e17b2e34-bc67-48f1-8198-c7cf24cd2858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:06.999963 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:07.074507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b2e34-bc67-48f1-8198-c7cf24cd2858 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:45:07.148798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c6812-7e0d-41c0-8e26-fbfae4374987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:45:07.149968 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c6812-7e0d-41c0-8e26-fbfae4374987 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:07.150248 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c6812-7e0d-41c0-8e26-fbfae4374987 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:07.150551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c6812-7e0d-41c0-8e26-fbfae4374987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:07.152172 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f79c6812-7e0d-41c0-8e26-fbfae4374987 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:07.249617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:07.286070 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c6812-7e0d-41c0-8e26-fbfae4374987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:45:07.287563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c6812-7e0d-41c0-8e26-fbfae4374987 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:45:07.306907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 653/1331] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:06 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:45:09.905656 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe6029-006f-488e-8a52-3bd67e14e4a6 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:09.906091 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe6029-006f-488e-8a52-3bd67e14e4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:09.906215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe6029-006f-488e-8a52-3bd67e14e4a6 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:09.907597 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 679/1332] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:09.912773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8231dc-d4c0-4a66-a45e-3ca5eb4f8263 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:09.913174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8231dc-d4c0-4a66-a45e-3ca5eb4f8263 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:09.913322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8231dc-d4c0-4a66-a45e-3ca5eb4f8263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:09.987081 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-1c8231dc-d4c0-4a66-a45e-3ca5eb4f8263 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:09.988274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1c8231dc-d4c0-4a66-a45e-3ca5eb4f8263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:10.080347 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 654/1333] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:45:10.116221 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe6029-006f-488e-8a52-3bd67e14e4a6 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:10.117761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-05fe6029-006f-488e-8a52-3bd67e14e4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:10.185695 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:10.219625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe6029-006f-488e-8a52-3bd67e14e4a6 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:45:10.290007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78360870-7623-4c44-8a02-6ac9b1a2d5d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:45:10.290652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78360870-7623-4c44-8a02-6ac9b1a2d5d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:10.290931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78360870-7623-4c44-8a02-6ac9b1a2d5d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:10.291200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78360870-7623-4c44-8a02-6ac9b1a2d5d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:10.292814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-78360870-7623-4c44-8a02-6ac9b1a2d5d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:10.398369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78360870-7623-4c44-8a02-6ac9b1a2d5d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:45:10.400349 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78360870-7623-4c44-8a02-6ac9b1a2d5d7 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:45:10.410078 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:10.418371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 680/1334] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:45:12.415226 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebd86c-49e1-4d12-af7f-fce3a1a7b01f None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:12.415764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebd86c-49e1-4d12-af7f-fce3a1a7b01f None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:12.415764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebd86c-49e1-4d12-af7f-fce3a1a7b01f None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:12.417106 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 655/1335] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:12.422101 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c590b6-039a-4211-bca0-1795c3ee0f5d None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:12.422521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c590b6-039a-4211-bca0-1795c3ee0f5d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:12.423198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c590b6-039a-4211-bca0-1795c3ee0f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:12.488566 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-b9c590b6-039a-4211-bca0-1795c3ee0f5d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:12.489772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b9c590b6-039a-4211-bca0-1795c3ee0f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:12.584246 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 681/1336] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:12.623971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebd86c-49e1-4d12-af7f-fce3a1a7b01f None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:12.625498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-4bebd86c-49e1-4d12-af7f-fce3a1a7b01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:12.665062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:12.728666 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebd86c-49e1-4d12-af7f-fce3a1a7b01f None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:45:12.800029 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a2e33-358e-47b7-abfa-bda1066af7cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:45:12.800640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a2e33-358e-47b7-abfa-bda1066af7cc service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:12.800921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a2e33-358e-47b7-abfa-bda1066af7cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:12.801194 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a2e33-358e-47b7-abfa-bda1066af7cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:12.802839 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6b1a2e33-358e-47b7-abfa-bda1066af7cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:12.884187 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:12.910106 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a2e33-358e-47b7-abfa-bda1066af7cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:45:12.911613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a2e33-358e-47b7-abfa-bda1066af7cc service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:45:12.929701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 656/1337] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:45:15.335816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40368c-19d8-4d69-b653-c0248aebaae6 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:15.336842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40368c-19d8-4d69-b653-c0248aebaae6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:15.336842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40368c-19d8-4d69-b653-c0248aebaae6 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:15.338198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 682/1338] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:15.343350 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b0ea0-dfbf-4f28-aec7-fab82a121239 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:15.343784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b0ea0-dfbf-4f28-aec7-fab82a121239 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:15.343914 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2b0ea0-dfbf-4f28-aec7-fab82a121239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:15.409374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-0e2b0ea0-dfbf-4f28-aec7-fab82a121239 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:15.410646 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-0e2b0ea0-dfbf-4f28-aec7-fab82a121239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:15.505188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 657/1339] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:15.541443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40368c-19d8-4d69-b653-c0248aebaae6 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:15.542990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-6e40368c-19d8-4d69-b653-c0248aebaae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:15.622237 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:15.650441 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40368c-19d8-4d69-b653-c0248aebaae6 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:45:15.720839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f16d4-9636-4239-9c62-0ab556d9d2ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:45:15.721449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f16d4-9636-4239-9c62-0ab556d9d2ad service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:15.721727 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f16d4-9636-4239-9c62-0ab556d9d2ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:15.721998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f16d4-9636-4239-9c62-0ab556d9d2ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:15.723638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-106f16d4-9636-4239-9c62-0ab556d9d2ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:15.828955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f16d4-9636-4239-9c62-0ab556d9d2ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:45:15.830374 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f16d4-9636-4239-9c62-0ab556d9d2ad service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:45:15.849352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 683/1340] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:45:19.817969 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad022a1-9b19-406f-89ff-dc47b72cf6cf None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:19.818619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad022a1-9b19-406f-89ff-dc47b72cf6cf None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:19.818619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad022a1-9b19-406f-89ff-dc47b72cf6cf None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:19.819905 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 658/1341] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:19.825633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f293bb-7adb-4dde-8002-2c7355c6237d None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:19.826051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f293bb-7adb-4dde-8002-2c7355c6237d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:19.826187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f293bb-7adb-4dde-8002-2c7355c6237d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:19.831651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:19.893808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-41f293bb-7adb-4dde-8002-2c7355c6237d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:19.895080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-41f293bb-7adb-4dde-8002-2c7355c6237d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:19.989729 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 684/1342] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:20.028142 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad022a1-9b19-406f-89ff-dc47b72cf6cf None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:20.029790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-cad022a1-9b19-406f-89ff-dc47b72cf6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:20.077610 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:20.133334 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad022a1-9b19-406f-89ff-dc47b72cf6cf None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:45:20.205129 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da5785d-7d4c-4f1d-aab0-cb4109d146ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:45:20.205747 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5785d-7d4c-4f1d-aab0-cb4109d146ca service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:20.206031 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5785d-7d4c-4f1d-aab0-cb4109d146ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:20.206307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da5785d-7d4c-4f1d-aab0-cb4109d146ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:20.207970 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-4da5785d-7d4c-4f1d-aab0-cb4109d146ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:20.300287 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:20.317093 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5785d-7d4c-4f1d-aab0-cb4109d146ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:45:20.318545 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da5785d-7d4c-4f1d-aab0-cb4109d146ca service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:45:20.336847 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 659/1343] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:45:22.932868 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc7fa8-d96a-456d-9999-2819769fba12 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:22.933439 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc7fa8-d96a-456d-9999-2819769fba12 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:22.933439 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc7fa8-d96a-456d-9999-2819769fba12 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:22.934823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 685/1344] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:22.940383 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d0c2d-b9d1-4dc9-b8f7-76d1eabfd811 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:22.940890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d0c2d-b9d1-4dc9-b8f7-76d1eabfd811 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:22.940986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d0c2d-b9d1-4dc9-b8f7-76d1eabfd811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:23.011660 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-1b9d0c2d-b9d1-4dc9-b8f7-76d1eabfd811 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:23.012883 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1b9d0c2d-b9d1-4dc9-b8f7-76d1eabfd811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:23.110943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 660/1345] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:23.148586 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc7fa8-d96a-456d-9999-2819769fba12 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:23.150149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-dacc7fa8-d96a-456d-9999-2819769fba12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:23.236278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:23.254451 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc7fa8-d96a-456d-9999-2819769fba12 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:45:23.335591 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89697fb2-c972-4e50-aa2a-c0d3600a48c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:45:23.336662 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89697fb2-c972-4e50-aa2a-c0d3600a48c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:23.336937 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89697fb2-c972-4e50-aa2a-c0d3600a48c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:23.337205 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89697fb2-c972-4e50-aa2a-c0d3600a48c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:23.338853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-89697fb2-c972-4e50-aa2a-c0d3600a48c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:23.448483 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89697fb2-c972-4e50-aa2a-c0d3600a48c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:45:23.449872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89697fb2-c972-4e50-aa2a-c0d3600a48c6 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:45:23.468525 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 686/1346] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:45:27.502030 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c8b6a-28e4-46a1-a17f-6a5a23019e13 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:27.502621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c8b6a-28e4-46a1-a17f-6a5a23019e13 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:27.502715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c8b6a-28e4-46a1-a17f-6a5a23019e13 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:27.504225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 661/1347] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:27.509654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ded1c-22cf-4745-8261-600ec06851a5 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:27.510084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ded1c-22cf-4745-8261-600ec06851a5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:27.510228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ded1c-22cf-4745-8261-600ec06851a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:27.526058 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:27.582163 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-d76ded1c-22cf-4745-8261-600ec06851a5 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:27.584289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d76ded1c-22cf-4745-8261-600ec06851a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:27.697404 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 687/1348] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:27.736125 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c8b6a-28e4-46a1-a17f-6a5a23019e13 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:27.737714 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-362c8b6a-28e4-46a1-a17f-6a5a23019e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:27.802762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:27.854159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c8b6a-28e4-46a1-a17f-6a5a23019e13 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:45:27.934930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afab5f-6b26-4d6b-9bd0-af252ab13563 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:45:27.935553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afab5f-6b26-4d6b-9bd0-af252ab13563 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:27.935835 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afab5f-6b26-4d6b-9bd0-af252ab13563 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:27.936111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afab5f-6b26-4d6b-9bd0-af252ab13563 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:27.937760 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-83afab5f-6b26-4d6b-9bd0-af252ab13563 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:28.048743 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:28.053423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afab5f-6b26-4d6b-9bd0-af252ab13563 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:45:28.055222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afab5f-6b26-4d6b-9bd0-af252ab13563 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:45:28.074565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 662/1349] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:45:32.884609 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ca72-fd5e-47b0-b0bf-1e961606addf None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:32.885541 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ca72-fd5e-47b0-b0bf-1e961606addf None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:32.886050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337ca72-fd5e-47b0-b0bf-1e961606addf None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:32.887917 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 688/1350] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 3 12:45:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:32.900105 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa5468-108f-4d5b-a07e-2ba1f6f2c5cb None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:32.901896 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa5468-108f-4d5b-a07e-2ba1f6f2c5cb None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:32.901896 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfa5468-108f-4d5b-a07e-2ba1f6f2c5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:33.043366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-5dfa5468-108f-4d5b-a07e-2ba1f6f2c5cb None None] MFA Rules not processed for user `5094263ac471481db48783b19192e01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:33.044695 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-5dfa5468-108f-4d5b-a07e-2ba1f6f2c5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:33.168332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 663/1351] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 3 12:45:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:37.213091 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc559da-c7ca-4896-ae0d-c5baafa5e462 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:37.213629 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc559da-c7ca-4896-ae0d-c5baafa5e462 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:37.213629 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc559da-c7ca-4896-ae0d-c5baafa5e462 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:37.215183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 689/1352] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 3 12:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:37.220129 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376a713-032a-4a3e-afc9-3662fc1fd1f7 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:37.220548 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376a713-032a-4a3e-afc9-3662fc1fd1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:37.220798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4376a713-032a-4a3e-afc9-3662fc1fd1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:37.278194 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:37.304055 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-4376a713-032a-4a3e-afc9-3662fc1fd1f7 None None] MFA Rules not processed for user `5094263ac471481db48783b19192e01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:37.306355 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-4376a713-032a-4a3e-afc9-3662fc1fd1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:37.432113 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 664/1353] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 3 12:45:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:37.694896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=59 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:38.808824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc06485-d781-44d3-ad88-96569f98f866 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:38.809444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc06485-d781-44d3-ad88-96569f98f866 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:38.809444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc06485-d781-44d3-ad88-96569f98f866 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:38.810737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 690/1354] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 3 12:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:38.815844 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e56b7-6a67-4a3a-814f-aee57567564b None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:38.816262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e56b7-6a67-4a3a-814f-aee57567564b None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:38.816406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e56b7-6a67-4a3a-814f-aee57567564b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:38.895588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-471e56b7-6a67-4a3a-814f-aee57567564b None None] MFA Rules not processed for user `5094263ac471481db48783b19192e01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:38.896831 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-471e56b7-6a67-4a3a-814f-aee57567564b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:38.938318 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:39.006371 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 665/1355] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 3 12:45:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:42.407772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6502bc9-e4f7-49e5-a638-c8a6c5dd936f None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:42.409390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6502bc9-e4f7-49e5-a638-c8a6c5dd936f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:42.409910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6502bc9-e4f7-49e5-a638-c8a6c5dd936f None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:42.415809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 691/1356] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:42.418011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390ce09-8f28-463c-8a83-804b33ea587a None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:42.418464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390ce09-8f28-463c-8a83-804b33ea587a None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:42.418583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390ce09-8f28-463c-8a83-804b33ea587a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:42.500050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-e390ce09-8f28-463c-8a83-804b33ea587a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:42.501403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e390ce09-8f28-463c-8a83-804b33ea587a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:42.605371 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 666/1357] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:42.659389 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e47917-d1c6-4c6b-b0fe-e7f31c70b996 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:42.660116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e47917-d1c6-4c6b-b0fe-e7f31c70b996 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:42.660548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e47917-d1c6-4c6b-b0fe-e7f31c70b996 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:42.662371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 692/1358] 10.222.0.65 () {54 vars in 962 bytes} [Wed Aug 3 12:45:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:42.670185 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5854f-d36c-4dd6-916a-a5e7a3480e71 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:42.670740 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5854f-d36c-4dd6-916a-a5e7a3480e71 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:42.671182 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5854f-d36c-4dd6-916a-a5e7a3480e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:42.691237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:42.758050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-8dc5854f-d36c-4dd6-916a-a5e7a3480e71 None None] MFA Rules not processed for user `7b634f567dca42dc95df175a4f36c1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:42.759368 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8dc5854f-d36c-4dd6-916a-a5e7a3480e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:42.889511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 667/1359] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Aug 3 12:45:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:42.897631 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e47917-d1c6-4c6b-b0fe-e7f31c70b996 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:42.899197 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e0e47917-d1c6-4c6b-b0fe-e7f31c70b996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:43.056154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e47917-d1c6-4c6b-b0fe-e7f31c70b996 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:45:43.132401 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:43.174519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e1f8c-0ebd-48e7-8171-e76574ef293f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:45:43.175150 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e1f8c-0ebd-48e7-8171-e76574ef293f service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:43.175431 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e1f8c-0ebd-48e7-8171-e76574ef293f service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:43.175707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e1f8c-0ebd-48e7-8171-e76574ef293f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:43.177479 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-4c4e1f8c-0ebd-48e7-8171-e76574ef293f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:43.271938 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e1f8c-0ebd-48e7-8171-e76574ef293f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:45:43.273594 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4e1f8c-0ebd-48e7-8171-e76574ef293f service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:45:43.292666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 693/1360] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:45:45.495469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b30b9-5756-495b-9b51-da74de29ba5d None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:45.495969 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b30b9-5756-495b-9b51-da74de29ba5d None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:45.496141 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b30b9-5756-495b-9b51-da74de29ba5d None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:45.497565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 668/1361] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:45.502588 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95952329-a446-40cb-a89b-2532930f5db9 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:45.503136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95952329-a446-40cb-a89b-2532930f5db9 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:45.503136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95952329-a446-40cb-a89b-2532930f5db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:45.573381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-95952329-a446-40cb-a89b-2532930f5db9 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:45.574534 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-95952329-a446-40cb-a89b-2532930f5db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:45.614224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:45.676958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 694/1362] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:45.721090 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b30b9-5756-495b-9b51-da74de29ba5d None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:45.723997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d62b30b9-5756-495b-9b51-da74de29ba5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:45.798329 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:45.897875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b30b9-5756-495b-9b51-da74de29ba5d None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:45:46.041054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ed90d-cab5-4186-ad16-08415b0d02a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:45:46.041701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ed90d-cab5-4186-ad16-08415b0d02a3 service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:46.041988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ed90d-cab5-4186-ad16-08415b0d02a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:46.042255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ed90d-cab5-4186-ad16-08415b0d02a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:46.043881 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-cd7ed90d-cab5-4186-ad16-08415b0d02a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:46.071515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:46.144564 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ed90d-cab5-4186-ad16-08415b0d02a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:45:46.146183 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ed90d-cab5-4186-ad16-08415b0d02a3 service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:45:46.165469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 669/1363] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:45:48.521327 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b485d9e-96ed-4974-9436-06b1b7fd8df0 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:48.521902 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b485d9e-96ed-4974-9436-06b1b7fd8df0 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:48.521902 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b485d9e-96ed-4974-9436-06b1b7fd8df0 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:48.523809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 695/1364] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:48.529151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ab180-547c-4132-bdea-718edd754715 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:48.529629 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ab180-547c-4132-bdea-718edd754715 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:48.529731 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ab180-547c-4132-bdea-718edd754715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:48.607331 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-6b0ab180-547c-4132-bdea-718edd754715 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:48.608577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6b0ab180-547c-4132-bdea-718edd754715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:48.720684 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:48.723619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 670/1365] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:48.771992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf6a25-d235-4dd1-9a80-2ebfa917d9a7 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:48.772277 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf6a25-d235-4dd1-9a80-2ebfa917d9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:48.772583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf6a25-d235-4dd1-9a80-2ebfa917d9a7 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:48.774046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 696/1366] 10.222.0.65 () {54 vars in 962 bytes} [Wed Aug 3 12:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:48.779580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a51e0-fcc6-4db8-901e-674bcd3260af None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:48.779867 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a51e0-fcc6-4db8-901e-674bcd3260af None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:48.780139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361a51e0-fcc6-4db8-901e-674bcd3260af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:48.863077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-361a51e0-fcc6-4db8-901e-674bcd3260af None None] MFA Rules not processed for user `7b634f567dca42dc95df175a4f36c1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:48.864911 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-361a51e0-fcc6-4db8-901e-674bcd3260af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:48.999134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 671/1367] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Aug 3 12:45:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:49.006726 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf6a25-d235-4dd1-9a80-2ebfa917d9a7 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:49.008756 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-4ecf6a25-d235-4dd1-9a80-2ebfa917d9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:49.166621 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf6a25-d235-4dd1-9a80-2ebfa917d9a7 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:45:49.187873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:49.285362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a80781-bd66-4534-9fdc-ccb307f3ae29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:45:49.286074 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a80781-bd66-4534-9fdc-ccb307f3ae29 service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:49.286420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a80781-bd66-4534-9fdc-ccb307f3ae29 service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:49.286785 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a80781-bd66-4534-9fdc-ccb307f3ae29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:49.288974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-50a80781-bd66-4534-9fdc-ccb307f3ae29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:49.395116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a80781-bd66-4534-9fdc-ccb307f3ae29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:45:49.396897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a80781-bd66-4534-9fdc-ccb307f3ae29 service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:45:49.416104 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 697/1368] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:45:50.451409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c44a9-fbbc-4521-9b78-1fcc3143769e None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:50.452016 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c44a9-fbbc-4521-9b78-1fcc3143769e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:50.452016 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c44a9-fbbc-4521-9b78-1fcc3143769e None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:50.453482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 672/1369] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:50.458616 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1a8ab-bf52-44f1-9d86-d43c92068fc7 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:50.459045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1a8ab-bf52-44f1-9d86-d43c92068fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:50.459162 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1a8ab-bf52-44f1-9d86-d43c92068fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:50.492327 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:50.528278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-a9d1a8ab-bf52-44f1-9d86-d43c92068fc7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:50.529483 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a9d1a8ab-bf52-44f1-9d86-d43c92068fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:50.634791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 698/1370] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:50.978162 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e92f20-a14e-467e-970e-c035d1dfb504 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:50.978162 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e92f20-a14e-467e-970e-c035d1dfb504 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:50.978162 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e92f20-a14e-467e-970e-c035d1dfb504 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:50.980666 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 673/1371] 10.222.0.65 () {54 vars in 964 bytes} [Wed Aug 3 12:45:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:50.985590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156eb7f-14f7-4b62-a08d-5440e4085619 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:50.985590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156eb7f-14f7-4b62-a08d-5440e4085619 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:50.985590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156eb7f-14f7-4b62-a08d-5440e4085619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:51.069276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-5156eb7f-14f7-4b62-a08d-5440e4085619 None None] MFA Rules not processed for user `473a5e7e980b4473acec85905fa7078c`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:51.070552 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5156eb7f-14f7-4b62-a08d-5440e4085619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:51.122804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:51.158329 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 699/1372] 10.222.0.65 () {58 vars in 1061 bytes} [Wed Aug 3 12:45:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:51.167674 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e92f20-a14e-467e-970e-c035d1dfb504 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:51.169317 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-b1e92f20-a14e-467e-970e-c035d1dfb504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:51.226262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:51.284007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e92f20-a14e-467e-970e-c035d1dfb504 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:45:51.379189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed99ec-1a9c-47c2-83d8-2cc29e3229e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:45:51.379817 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed99ec-1a9c-47c2-83d8-2cc29e3229e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:51.380153 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed99ec-1a9c-47c2-83d8-2cc29e3229e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:51.380436 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed99ec-1a9c-47c2-83d8-2cc29e3229e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:51.382686 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-3bed99ec-1a9c-47c2-83d8-2cc29e3229e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:51.491153 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:51.499615 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac0ab5-f9d3-47cf-b181-55438cdaa193 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:51.500134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac0ab5-f9d3-47cf-b181-55438cdaa193 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:51.500230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac0ab5-f9d3-47cf-b181-55438cdaa193 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:51.501499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 700/1373] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:51.506451 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802c012-77b1-425f-816c-dee583d14823 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:51.506769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802c012-77b1-425f-816c-dee583d14823 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:51.507618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802c012-77b1-425f-816c-dee583d14823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:51.509254 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed99ec-1a9c-47c2-83d8-2cc29e3229e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:45:51.511472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed99ec-1a9c-47c2-83d8-2cc29e3229e0 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:45:51.533902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 674/1374] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:45:51.575214 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-0802c012-77b1-425f-816c-dee583d14823 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:51.576405 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0802c012-77b1-425f-816c-dee583d14823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:51.674576 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 701/1375] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:45:51.723139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed99ec-1a9c-47c2-83d8-2cc29e3229e0 service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:51.725077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-3bed99ec-1a9c-47c2-83d8-2cc29e3229e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:51.829416 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed99ec-1a9c-47c2-83d8-2cc29e3229e0 service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:45:51.938111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:51.951077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fdd99d-004a-45a2-844b-7a9bd911de4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:45:51.951700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdd99d-004a-45a2-844b-7a9bd911de4f service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:51.952038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdd99d-004a-45a2-844b-7a9bd911de4f service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:51.952314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdd99d-004a-45a2-844b-7a9bd911de4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:51.953988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e5fdd99d-004a-45a2-844b-7a9bd911de4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:52.041834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fdd99d-004a-45a2-844b-7a9bd911de4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:45:52.043463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fdd99d-004a-45a2-844b-7a9bd911de4f service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:45:52.062040 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 675/1376] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:45:54.083718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28cde1-2c32-402f-b70d-2e2b80ab4f28 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:54.084233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28cde1-2c32-402f-b70d-2e2b80ab4f28 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:54.084355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28cde1-2c32-402f-b70d-2e2b80ab4f28 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:54.086169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 702/1377] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:54.091524 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0dd39-593a-43ed-94df-053b40af96bd None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:54.091993 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0dd39-593a-43ed-94df-053b40af96bd None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:54.092092 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d0dd39-593a-43ed-94df-053b40af96bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:54.158174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-f9d0dd39-593a-43ed-94df-053b40af96bd None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:54.159315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f9d0dd39-593a-43ed-94df-053b40af96bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:54.219905 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:54.259310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 676/1378] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:54.303344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28cde1-2c32-402f-b70d-2e2b80ab4f28 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:54.305152 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-2c28cde1-2c32-402f-b70d-2e2b80ab4f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:54.357970 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:54.586533 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28cde1-2c32-402f-b70d-2e2b80ab4f28 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:45:54.698980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e3764-8e74-4564-a7de-15375e0e7605 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:45:54.699595 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e3764-8e74-4564-a7de-15375e0e7605 service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:54.699869 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e3764-8e74-4564-a7de-15375e0e7605 service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:54.700133 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e3764-8e74-4564-a7de-15375e0e7605 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:54.701732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-080e3764-8e74-4564-a7de-15375e0e7605 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:54.713603 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:54.794707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e3764-8e74-4564-a7de-15375e0e7605 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:45:54.796295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e3764-8e74-4564-a7de-15375e0e7605 service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:45:54.814929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 703/1379] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:45:56.890582 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a8a73-2df9-4e4a-8f6f-a43f317eb8f7 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:56.891062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a8a73-2df9-4e4a-8f6f-a43f317eb8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:56.891062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a8a73-2df9-4e4a-8f6f-a43f317eb8f7 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:56.892573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 677/1380] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:56.897646 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3c488-c3f4-4dbd-b201-0283797d4dc1 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:56.898090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3c488-c3f4-4dbd-b201-0283797d4dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:56.898189 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3c488-c3f4-4dbd-b201-0283797d4dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:56.968816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-cba3c488-c3f4-4dbd-b201-0283797d4dc1 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:56.970151 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-cba3c488-c3f4-4dbd-b201-0283797d4dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:57.076879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:57.092413 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 704/1381] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:57.136272 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a8a73-2df9-4e4a-8f6f-a43f317eb8f7 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:57.137858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-703a8a73-2df9-4e4a-8f6f-a43f317eb8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:57.293250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a8a73-2df9-4e4a-8f6f-a43f317eb8f7 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:45:57.354107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:57.423207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560534f2-6770-4a6a-9c24-666b3c708132 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:45:57.423835 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560534f2-6770-4a6a-9c24-666b3c708132 service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:57.424115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560534f2-6770-4a6a-9c24-666b3c708132 service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:57.424391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560534f2-6770-4a6a-9c24-666b3c708132 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:57.426083 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-560534f2-6770-4a6a-9c24-666b3c708132 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:57.527279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560534f2-6770-4a6a-9c24-666b3c708132 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:45:57.529401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560534f2-6770-4a6a-9c24-666b3c708132 service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:45:57.548717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 678/1382] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:45:59.627289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c564a2-aaa6-4ce3-9f30-95f4567e6aaf None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:59.627998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c564a2-aaa6-4ce3-9f30-95f4567e6aaf None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:59.627998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c564a2-aaa6-4ce3-9f30-95f4567e6aaf None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:59.629471 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 705/1383] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:45:59.635786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7832800-d2d1-4e11-99cf-cb1fb3a3adaf None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:45:59.636209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7832800-d2d1-4e11-99cf-cb1fb3a3adaf None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:45:59.636541 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7832800-d2d1-4e11-99cf-cb1fb3a3adaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:45:59.713078 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-b7832800-d2d1-4e11-99cf-cb1fb3a3adaf None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:45:59.714596 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-b7832800-d2d1-4e11-99cf-cb1fb3a3adaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:45:59.716859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:45:59.823908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 679/1384] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:45:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:45:59.866003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c564a2-aaa6-4ce3-9f30-95f4567e6aaf None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:45:59.867618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-41c564a2-aaa6-4ce3-9f30-95f4567e6aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:00.015172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c564a2-aaa6-4ce3-9f30-95f4567e6aaf None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:00.082507 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:00.134739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba4418-26e2-4e0d-908c-0b2f4277fc6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:00.135281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba4418-26e2-4e0d-908c-0b2f4277fc6a service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:00.135483 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba4418-26e2-4e0d-908c-0b2f4277fc6a service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:00.135755 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba4418-26e2-4e0d-908c-0b2f4277fc6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:00.137409 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f0ba4418-26e2-4e0d-908c-0b2f4277fc6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:00.229742 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba4418-26e2-4e0d-908c-0b2f4277fc6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:00.231310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba4418-26e2-4e0d-908c-0b2f4277fc6a service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:00.249173 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 706/1385] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:45:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:02.820552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60238129-4895-4598-9de3-eb1ef98f1329 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:02.821141 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60238129-4895-4598-9de3-eb1ef98f1329 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:02.821241 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60238129-4895-4598-9de3-eb1ef98f1329 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:02.823055 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 680/1386] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:46:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:02.828592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca5a39-b2fa-4355-872f-b76760b21f19 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:02.829003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca5a39-b2fa-4355-872f-b76760b21f19 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:02.829337 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca5a39-b2fa-4355-872f-b76760b21f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:02.908308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-aaca5a39-b2fa-4355-872f-b76760b21f19 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:02.909875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-aaca5a39-b2fa-4355-872f-b76760b21f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:02.936671 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:03.019301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 707/1387] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:46:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:46:03.063301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60238129-4895-4598-9de3-eb1ef98f1329 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:03.065192 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-60238129-4895-4598-9de3-eb1ef98f1329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:03.210120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:03.226969 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60238129-4895-4598-9de3-eb1ef98f1329 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:03.345755 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094121f-453c-41fa-96f7-c7e9130273df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:03.346417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094121f-453c-41fa-96f7-c7e9130273df service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:03.346744 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094121f-453c-41fa-96f7-c7e9130273df service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:03.347035 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094121f-453c-41fa-96f7-c7e9130273df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:03.348795 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-3094121f-453c-41fa-96f7-c7e9130273df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:03.442733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094121f-453c-41fa-96f7-c7e9130273df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:03.444453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094121f-453c-41fa-96f7-c7e9130273df service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:03.450198 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:03.464995 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 681/1388] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:05.472078 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef124188-beb4-44e2-9fc7-45c40031f6ea None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:05.472706 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef124188-beb4-44e2-9fc7-45c40031f6ea None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:05.472706 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef124188-beb4-44e2-9fc7-45c40031f6ea None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:05.473968 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 708/1389] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:05.479863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4697bd9-c9c3-4dcd-a53e-74b90da7763a None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:05.480259 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4697bd9-c9c3-4dcd-a53e-74b90da7763a None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:05.480424 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4697bd9-c9c3-4dcd-a53e-74b90da7763a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:05.545373 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a4697bd9-c9c3-4dcd-a53e-74b90da7763a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:05.546623 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a4697bd9-c9c3-4dcd-a53e-74b90da7763a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:05.645561 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 682/1390] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:46:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:05.691108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef124188-beb4-44e2-9fc7-45c40031f6ea None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:05.692700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ef124188-beb4-44e2-9fc7-45c40031f6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:05.844822 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef124188-beb4-44e2-9fc7-45c40031f6ea None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:05.854220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:05.957297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735a633-c502-495a-a7b5-66e350bc817d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:05.957911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735a633-c502-495a-a7b5-66e350bc817d service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:05.958187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735a633-c502-495a-a7b5-66e350bc817d service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:05.958460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735a633-c502-495a-a7b5-66e350bc817d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:05.960713 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f735a633-c502-495a-a7b5-66e350bc817d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:06.046784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735a633-c502-495a-a7b5-66e350bc817d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:06.048354 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735a633-c502-495a-a7b5-66e350bc817d service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:06.066801 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 709/1391] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:06.661934 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d568fac-3b94-4d26-951a-0df7a983e597 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:06.663021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d568fac-3b94-4d26-951a-0df7a983e597 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:06.663021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d568fac-3b94-4d26-951a-0df7a983e597 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:06.664377 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 683/1392] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:46:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:06.669396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb1b45-1fac-4d41-9e49-a6c9309d00eb None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:06.669854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb1b45-1fac-4d41-9e49-a6c9309d00eb None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:06.669953 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb1b45-1fac-4d41-9e49-a6c9309d00eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:06.685896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:06.736606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-a1eb1b45-1fac-4d41-9e49-a6c9309d00eb None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:06.737822 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a1eb1b45-1fac-4d41-9e49-a6c9309d00eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:06.833775 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 710/1393] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:46:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:06.892317 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76528801-8b1a-447f-ab8a-ec53cc62aeea None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:06.892608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76528801-8b1a-447f-ab8a-ec53cc62aeea None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:06.892894 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76528801-8b1a-447f-ab8a-ec53cc62aeea None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:06.894240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 684/1394] 10.222.0.65 () {54 vars in 966 bytes} [Wed Aug 3 12:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:06.899854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ec7ac-e570-4d2a-a4f7-0e282d8ce59f None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:06.900149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ec7ac-e570-4d2a-a4f7-0e282d8ce59f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:06.900410 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ec7ac-e570-4d2a-a4f7-0e282d8ce59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:06.978308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-a62ec7ac-e570-4d2a-a4f7-0e282d8ce59f None None] MFA Rules not processed for user `e6066e48aa644a47a889e09cfe204858`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:06.979559 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a62ec7ac-e570-4d2a-a4f7-0e282d8ce59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:07.013088 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:07.054957 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 711/1395] 10.222.0.65 () {58 vars in 1063 bytes} [Wed Aug 3 12:46:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:07.062661 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76528801-8b1a-447f-ab8a-ec53cc62aeea None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:07.064259 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-76528801-8b1a-447f-ab8a-ec53cc62aeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:07.139655 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:07.171104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76528801-8b1a-447f-ab8a-ec53cc62aeea None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:07.268232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07128198-3da1-4c2b-a33a-d70b95393914 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6066e48aa644a47a889e09cfe204858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:07.268764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07128198-3da1-4c2b-a33a-d70b95393914 service glance] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:07.269043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07128198-3da1-4c2b-a33a-d70b95393914 service glance] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:07.269316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07128198-3da1-4c2b-a33a-d70b95393914 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:07.270979 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-07128198-3da1-4c2b-a33a-d70b95393914 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:07.378193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07128198-3da1-4c2b-a33a-d70b95393914 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:07.379649 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07128198-3da1-4c2b-a33a-d70b95393914 service glance] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:07.395290 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:07.398834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 685/1396] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:08.114466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8ecc4-5af3-442e-b446-0f150600ac00 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:08.115118 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8ecc4-5af3-442e-b446-0f150600ac00 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:08.115118 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8ecc4-5af3-442e-b446-0f150600ac00 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:08.117115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 712/1397] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:46:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:46:08.122532 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4bf48-cc84-4efc-b4c3-359249dc721b None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:08.123256 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4bf48-cc84-4efc-b4c3-359249dc721b None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:08.123256 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4bf48-cc84-4efc-b4c3-359249dc721b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:08.130093 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be283d59-8811-4fd3-aeca-9b9ec4d9638f None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:08.130317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be283d59-8811-4fd3-aeca-9b9ec4d9638f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:08.130626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be283d59-8811-4fd3-aeca-9b9ec4d9638f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:08.188766 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a7a4bf48-cc84-4efc-b4c3-359249dc721b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:08.189974 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a7a4bf48-cc84-4efc-b4c3-359249dc721b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:08.197076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-be283d59-8811-4fd3-aeca-9b9ec4d9638f None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:08.198435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-be283d59-8811-4fd3-aeca-9b9ec4d9638f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:08.285672 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 713/1398] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 3 12:46:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:46:08.289238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 686/1399] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:46:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:08.332127 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e9341-d0dc-48da-8007-6dfea1f7d918 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:08.332416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e9341-d0dc-48da-8007-6dfea1f7d918 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:08.332696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e9341-d0dc-48da-8007-6dfea1f7d918 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:08.334040 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 714/1400] 10.222.0.65 () {54 vars in 943 bytes} [Wed Aug 3 12:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:08.335335 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4bf48-cc84-4efc-b4c3-359249dc721b None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:08.336664 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a7a4bf48-cc84-4efc-b4c3-359249dc721b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:08.340881 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75231926-a0d9-4f81-9fa9-cede341a1b40 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:08.341169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75231926-a0d9-4f81-9fa9-cede341a1b40 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:08.341566 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75231926-a0d9-4f81-9fa9-cede341a1b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:08.400177 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:08.413672 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:08.419446 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-75231926-a0d9-4f81-9fa9-cede341a1b40 None None] MFA Rules not processed for user `cea54cb772344612aec4686d484de89f`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:08.420670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-75231926-a0d9-4f81-9fa9-cede341a1b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:08.452329 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4bf48-cc84-4efc-b4c3-359249dc721b None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:08.527597 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 715/1401] 10.222.0.65 () {58 vars in 1040 bytes} [Wed Aug 3 12:46:08 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:46:08.537055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75231926-a0d9-4f81-9fa9-cede341a1b40 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:08.547325 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75231926-a0d9-4f81-9fa9-cede341a1b40 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:08.567542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed374e3-1e07-4d69-bacf-9de3f09c6576 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:08.568163 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed374e3-1e07-4d69-bacf-9de3f09c6576 service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:08.568441 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed374e3-1e07-4d69-bacf-9de3f09c6576 service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:08.568736 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed374e3-1e07-4d69-bacf-9de3f09c6576 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:08.570446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2ed374e3-1e07-4d69-bacf-9de3f09c6576 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:08.651259 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:08.651447 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:08.657193 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337404d-202e-4caa-b794-c35989d2398a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:08.657954 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337404d-202e-4caa-b794-c35989d2398a service swift] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:08.658241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337404d-202e-4caa-b794-c35989d2398a service swift] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:08.658544 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337404d-202e-4caa-b794-c35989d2398a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:08.660244 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-2337404d-202e-4caa-b794-c35989d2398a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:08.668763 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed374e3-1e07-4d69-bacf-9de3f09c6576 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:08.670361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed374e3-1e07-4d69-bacf-9de3f09c6576 service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:08.689444 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 687/1402] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:08.745406 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337404d-202e-4caa-b794-c35989d2398a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:08.747345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337404d-202e-4caa-b794-c35989d2398a service swift] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:08.764297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 716/1403] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:46:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 03 12:46:10.720521 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb258b87-00dd-4fd7-98f7-eba44fe42767 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:10.721095 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb258b87-00dd-4fd7-98f7-eba44fe42767 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:10.721095 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb258b87-00dd-4fd7-98f7-eba44fe42767 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:10.722518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 688/1404] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:10.727596 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9c6fa-ad71-4029-becc-38cf088ac9f7 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:10.728048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9c6fa-ad71-4029-becc-38cf088ac9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:10.728153 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc9c6fa-ad71-4029-becc-38cf088ac9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:10.809353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-bdc9c6fa-ad71-4029-becc-38cf088ac9f7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:10.811208 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-bdc9c6fa-ad71-4029-becc-38cf088ac9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:10.901718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:10.916423 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 717/1405] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:10.962088 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb258b87-00dd-4fd7-98f7-eba44fe42767 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:10.963609 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-fb258b87-00dd-4fd7-98f7-eba44fe42767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:11.130432 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb258b87-00dd-4fd7-98f7-eba44fe42767 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:11.187421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:11.257937 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2ffd0-6d81-4b79-ab6e-e3dcab869fba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:11.258464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2ffd0-6d81-4b79-ab6e-e3dcab869fba service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:11.258773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2ffd0-6d81-4b79-ab6e-e3dcab869fba service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:11.259066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2ffd0-6d81-4b79-ab6e-e3dcab869fba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:11.260809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-61a2ffd0-6d81-4b79-ab6e-e3dcab869fba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:11.355225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2ffd0-6d81-4b79-ab6e-e3dcab869fba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:11.356819 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2ffd0-6d81-4b79-ab6e-e3dcab869fba service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:11.375613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 689/1406] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:13.429743 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61407346-47bf-4ce4-9622-a9055d92b3c8 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:13.430304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61407346-47bf-4ce4-9622-a9055d92b3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:13.430304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61407346-47bf-4ce4-9622-a9055d92b3c8 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:13.432493 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 718/1407] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:46:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:13.438077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fecc55-8cbd-4935-a5eb-c48fb37c6294 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:13.438590 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fecc55-8cbd-4935-a5eb-c48fb37c6294 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:13.438674 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fecc55-8cbd-4935-a5eb-c48fb37c6294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:13.516469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a9fecc55-8cbd-4935-a5eb-c48fb37c6294 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:13.518317 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a9fecc55-8cbd-4935-a5eb-c48fb37c6294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:13.527215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:13.626420 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 690/1408] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:13.671227 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61407346-47bf-4ce4-9622-a9055d92b3c8 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:13.672797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-61407346-47bf-4ce4-9622-a9055d92b3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:13.831417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61407346-47bf-4ce4-9622-a9055d92b3c8 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:13.910127 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:13.969343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6ce0f-1821-4269-b137-55ad32fcf763 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:13.969994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6ce0f-1821-4269-b137-55ad32fcf763 service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:13.970270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6ce0f-1821-4269-b137-55ad32fcf763 service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:13.970573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6ce0f-1821-4269-b137-55ad32fcf763 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:13.972178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c5c6ce0f-1821-4269-b137-55ad32fcf763 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:14.076776 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6ce0f-1821-4269-b137-55ad32fcf763 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:14.078352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c6ce0f-1821-4269-b137-55ad32fcf763 service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:14.099724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 719/1409] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:23.624660 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=39 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:24.097092 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=67 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:30.738048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08650543-69f2-4947-afd5-121eb21d41b6 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:30.738650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08650543-69f2-4947-afd5-121eb21d41b6 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:30.738650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08650543-69f2-4947-afd5-121eb21d41b6 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:30.740563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 691/1410] 10.222.0.65 () {54 vars in 918 bytes} [Wed Aug 3 12:46:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:30.745459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c5871-8a46-4e72-9253-8aec4718b9b5 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:30.745909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c5871-8a46-4e72-9253-8aec4718b9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:30.746017 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c5871-8a46-4e72-9253-8aec4718b9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:30.824826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-b81c5871-8a46-4e72-9253-8aec4718b9b5 None None] MFA Rules not processed for user `5094263ac471481db48783b19192e01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:30.826052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b81c5871-8a46-4e72-9253-8aec4718b9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:30.929954 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 720/1411] 10.222.0.65 () {58 vars in 1015 bytes} [Wed Aug 3 12:46:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:30.944036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330fabe-7246-4886-95d1-0333d4b8a687 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:30.944318 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330fabe-7246-4886-95d1-0333d4b8a687 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:30.944587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330fabe-7246-4886-95d1-0333d4b8a687 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:30.945967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 692/1412] 10.222.0.65 () {54 vars in 918 bytes} [Wed Aug 3 12:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:30.951079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b134a1-7ab2-45b3-88a9-b5c96ac3328a None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:30.951360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b134a1-7ab2-45b3-88a9-b5c96ac3328a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:30.951629 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b134a1-7ab2-45b3-88a9-b5c96ac3328a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:31.022221 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:31.024848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-39b134a1-7ab2-45b3-88a9-b5c96ac3328a None None] MFA Rules not processed for user `7b634f567dca42dc95df175a4f36c1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:31.026632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-39b134a1-7ab2-45b3-88a9-b5c96ac3328a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:31.141905 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 721/1413] 10.222.0.65 () {58 vars in 1015 bytes} [Wed Aug 3 12:46:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:46:33.583511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddbadf-222e-430d-8c79-9040fd463632 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:33.584561 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddbadf-222e-430d-8c79-9040fd463632 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:33.584561 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaddbadf-222e-430d-8c79-9040fd463632 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:33.586085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 693/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 12:46:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:33.591454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699675db-4805-4992-b7c7-bc93feb15c2c None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:33.591847 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699675db-4805-4992-b7c7-bc93feb15c2c None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:33.592011 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699675db-4805-4992-b7c7-bc93feb15c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:33.658418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-699675db-4805-4992-b7c7-bc93feb15c2c None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:33.659857 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-699675db-4805-4992-b7c7-bc93feb15c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:33.714003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:33.754941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 722/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 12:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:33.797483 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddbadf-222e-430d-8c79-9040fd463632 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:33.799079 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-aaddbadf-222e-430d-8c79-9040fd463632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:33.944703 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaddbadf-222e-430d-8c79-9040fd463632 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:34.016530 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:34.057056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66523861-7bc2-4391-a37f-859988816842 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:34.057673 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66523861-7bc2-4391-a37f-859988816842 service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:34.057952 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66523861-7bc2-4391-a37f-859988816842 service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:34.058222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66523861-7bc2-4391-a37f-859988816842 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:34.059869 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-66523861-7bc2-4391-a37f-859988816842 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:34.146697 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66523861-7bc2-4391-a37f-859988816842 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:34.148279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66523861-7bc2-4391-a37f-859988816842 service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:34.166092 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 694/1416] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:35.073867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d57336-ac9e-4028-896b-838f6e71993b None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:35.074412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d57336-ac9e-4028-896b-838f6e71993b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:35.074412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d57336-ac9e-4028-896b-838f6e71993b None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:35.075809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 723/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 12:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:35.080733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f97c6-80e6-4d8a-965f-51c99333a08a None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:35.081160 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f97c6-80e6-4d8a-965f-51c99333a08a None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:35.081308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f97c6-80e6-4d8a-965f-51c99333a08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:35.157810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-ab4f97c6-80e6-4d8a-965f-51c99333a08a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:35.158988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ab4f97c6-80e6-4d8a-965f-51c99333a08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:35.185383 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:35.255074 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 695/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 12:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:35.405629 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d57336-ac9e-4028-896b-838f6e71993b None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:35.407164 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-43d57336-ac9e-4028-896b-838f6e71993b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:35.549170 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d57336-ac9e-4028-896b-838f6e71993b None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:35.589347 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:35.660130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a6506c-8d24-44a0-8de5-871bb6b862bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:35.660746 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6506c-8d24-44a0-8de5-871bb6b862bf service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:35.661020 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6506c-8d24-44a0-8de5-871bb6b862bf service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:35.661290 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a6506c-8d24-44a0-8de5-871bb6b862bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:35.663413 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-27a6506c-8d24-44a0-8de5-871bb6b862bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:35.750360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6506c-8d24-44a0-8de5-871bb6b862bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:35.751954 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a6506c-8d24-44a0-8de5-871bb6b862bf service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:35.769363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 724/1419] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:36.646573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbb27a-417a-41ca-85cc-c7ea575c3a26 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:36.647114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbb27a-417a-41ca-85cc-c7ea575c3a26 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:36.647114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbb27a-417a-41ca-85cc-c7ea575c3a26 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:36.648465 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 696/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 12:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:36.653411 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72b09d-35d7-4a3a-81b1-9fcd2add3a05 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:36.653863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72b09d-35d7-4a3a-81b1-9fcd2add3a05 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:36.653972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72b09d-35d7-4a3a-81b1-9fcd2add3a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:36.709380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:36.731475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-cd72b09d-35d7-4a3a-81b1-9fcd2add3a05 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:36.732694 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-cd72b09d-35d7-4a3a-81b1-9fcd2add3a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:36.826826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 725/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 12:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:36.978023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbb27a-417a-41ca-85cc-c7ea575c3a26 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:36.980017 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-3cbbb27a-417a-41ca-85cc-c7ea575c3a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:37.122134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbb27a-417a-41ca-85cc-c7ea575c3a26 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:37.134612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:37.234006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3668a2-fea2-4325-9a9b-ef56ab67bc1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:37.234598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3668a2-fea2-4325-9a9b-ef56ab67bc1e service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:37.234879 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3668a2-fea2-4325-9a9b-ef56ab67bc1e service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:37.235149 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3668a2-fea2-4325-9a9b-ef56ab67bc1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:37.236764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-cc3668a2-fea2-4325-9a9b-ef56ab67bc1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:37.323553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3668a2-fea2-4325-9a9b-ef56ab67bc1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:37.325115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3668a2-fea2-4325-9a9b-ef56ab67bc1e service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:37.342982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 697/1422] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:38.193291 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3992294-b676-4702-9261-e1e642d2a201 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:38.193904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3992294-b676-4702-9261-e1e642d2a201 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:38.193904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3992294-b676-4702-9261-e1e642d2a201 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:38.195207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 726/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 12:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:38.200108 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbda091-9245-47a8-b9c8-368200171820 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:38.200523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbda091-9245-47a8-b9c8-368200171820 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:38.200659 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbda091-9245-47a8-b9c8-368200171820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:38.219905 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:38.276602 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-3bbda091-9245-47a8-b9c8-368200171820 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:38.277785 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-3bbda091-9245-47a8-b9c8-368200171820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:38.371712 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 698/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 12:46:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:38.520499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3992294-b676-4702-9261-e1e642d2a201 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:38.522550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a3992294-b676-4702-9261-e1e642d2a201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:38.629686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:38.667176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3992294-b676-4702-9261-e1e642d2a201 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:38.777382 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221a305-5728-42bd-9140-c62b5e532eb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:38.777920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221a305-5728-42bd-9140-c62b5e532eb9 service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:38.778200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221a305-5728-42bd-9140-c62b5e532eb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:38.778471 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221a305-5728-42bd-9140-c62b5e532eb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:38.780110 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5221a305-5728-42bd-9140-c62b5e532eb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:38.853570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:38.867235 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221a305-5728-42bd-9140-c62b5e532eb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:38.868806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221a305-5728-42bd-9140-c62b5e532eb9 service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:38.886886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 727/1425] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:39.745540 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ece99-17bb-422e-b3ef-e30416338aed None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:39.746665 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ece99-17bb-422e-b3ef-e30416338aed None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:39.746943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ece99-17bb-422e-b3ef-e30416338aed None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:39.748311 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 699/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 12:46:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:39.753176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934eedb8-41e6-472e-80b2-331d1eb32ee4 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:39.753591 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934eedb8-41e6-472e-80b2-331d1eb32ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:39.753704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934eedb8-41e6-472e-80b2-331d1eb32ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:39.827918 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-934eedb8-41e6-472e-80b2-331d1eb32ee4 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:39.829105 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-934eedb8-41e6-472e-80b2-331d1eb32ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:39.924599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 728/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 12:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:40.074895 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ece99-17bb-422e-b3ef-e30416338aed None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:40.076330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c18ece99-17bb-422e-b3ef-e30416338aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:40.158608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:40.219751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ece99-17bb-422e-b3ef-e30416338aed None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:40.341861 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7446d8e-5970-47a3-acfd-797a71cba911 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:40.342402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7446d8e-5970-47a3-acfd-797a71cba911 service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:40.342715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7446d8e-5970-47a3-acfd-797a71cba911 service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:40.342986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7446d8e-5970-47a3-acfd-797a71cba911 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:40.344614 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f7446d8e-5970-47a3-acfd-797a71cba911 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:40.391973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:40.433048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7446d8e-5970-47a3-acfd-797a71cba911 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:40.434671 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7446d8e-5970-47a3-acfd-797a71cba911 service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:40.452870 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 700/1428] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:41.250937 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aab74e-1d17-4673-bb4f-43ed9b957851 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:41.251463 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aab74e-1d17-4673-bb4f-43ed9b957851 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:41.251593 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aab74e-1d17-4673-bb4f-43ed9b957851 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:41.252872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 729/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 12:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:41.257981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bd799-7aaa-4766-9dcf-cb1b274d87d6 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:41.258435 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bd799-7aaa-4766-9dcf-cb1b274d87d6 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:41.258581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bd799-7aaa-4766-9dcf-cb1b274d87d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:41.335011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-d40bd799-7aaa-4766-9dcf-cb1b274d87d6 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:41.336214 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d40bd799-7aaa-4766-9dcf-cb1b274d87d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:41.550360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 701/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 12:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:41.700346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aab74e-1d17-4673-bb4f-43ed9b957851 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:41.701894 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-18aab74e-1d17-4673-bb4f-43ed9b957851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:41.749682 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:41.846135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aab74e-1d17-4673-bb4f-43ed9b957851 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:41.956867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e2fbb3-97be-4cf5-8245-98f60a50441e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:41.957482 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2fbb3-97be-4cf5-8245-98f60a50441e service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:41.957781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2fbb3-97be-4cf5-8245-98f60a50441e service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:41.958049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e2fbb3-97be-4cf5-8245-98f60a50441e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:41.960330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-09e2fbb3-97be-4cf5-8245-98f60a50441e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:41.967084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:42.047113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2fbb3-97be-4cf5-8245-98f60a50441e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:42.048667 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e2fbb3-97be-4cf5-8245-98f60a50441e service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:42.068355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 730/1431] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:42.868111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c964f7-d265-444c-a688-fd4112e4983c None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:42.868679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c964f7-d265-444c-a688-fd4112e4983c None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:42.868679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c964f7-d265-444c-a688-fd4112e4983c None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:42.870020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 702/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 12:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:42.875021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ebba0-0e4c-4e83-acf0-9bebcacce481 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:42.875425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ebba0-0e4c-4e83-acf0-9bebcacce481 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:42.875585 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ebba0-0e4c-4e83-acf0-9bebcacce481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:42.951211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-372ebba0-0e4c-4e83-acf0-9bebcacce481 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:42.952416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-372ebba0-0e4c-4e83-acf0-9bebcacce481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:43.019622 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:43.049299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 731/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 12:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:43.201656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c964f7-d265-444c-a688-fd4112e4983c None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:43.203138 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-71c964f7-d265-444c-a688-fd4112e4983c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:43.220936 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:43.345184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c964f7-d265-444c-a688-fd4112e4983c None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:43.429282 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:43.455549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204960da-73ea-4db4-b59a-a55ed7b582cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:43.456092 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204960da-73ea-4db4-b59a-a55ed7b582cb service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:43.456369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204960da-73ea-4db4-b59a-a55ed7b582cb service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:43.456638 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204960da-73ea-4db4-b59a-a55ed7b582cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:43.458939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-204960da-73ea-4db4-b59a-a55ed7b582cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:43.545148 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204960da-73ea-4db4-b59a-a55ed7b582cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:43.546774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204960da-73ea-4db4-b59a-a55ed7b582cb service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:43.564850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 703/1434] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:44.362155 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897f493-0b6b-490c-9236-aba3e878c462 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:44.362754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897f493-0b6b-490c-9236-aba3e878c462 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:44.362754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897f493-0b6b-490c-9236-aba3e878c462 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:44.364076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 732/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 12:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:44.368924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a194ba-c7be-462d-84a5-e48764d5be6d None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:44.369367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a194ba-c7be-462d-84a5-e48764d5be6d None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:44.369498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a194ba-c7be-462d-84a5-e48764d5be6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:44.444567 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-c1a194ba-c7be-462d-84a5-e48764d5be6d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:44.445791 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c1a194ba-c7be-462d-84a5-e48764d5be6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:44.487130 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:44.541541 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 704/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 12:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:44.692386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897f493-0b6b-490c-9236-aba3e878c462 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:44.694341 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e897f493-0b6b-490c-9236-aba3e878c462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:44.835243 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897f493-0b6b-490c-9236-aba3e878c462 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:44.887431 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:44.946770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2975dcd-7867-45b4-b4fb-7ae757ba38d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:44.947323 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2975dcd-7867-45b4-b4fb-7ae757ba38d5 service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:44.947599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2975dcd-7867-45b4-b4fb-7ae757ba38d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:44.947868 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2975dcd-7867-45b4-b4fb-7ae757ba38d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:44.949786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e2975dcd-7867-45b4-b4fb-7ae757ba38d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:45.037014 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2975dcd-7867-45b4-b4fb-7ae757ba38d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:45.038657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2975dcd-7867-45b4-b4fb-7ae757ba38d5 service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:45.057160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 733/1437] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:45.858787 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3ea42-e7d5-405a-9ca6-2f90ff85cc0e None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:45.859327 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3ea42-e7d5-405a-9ca6-2f90ff85cc0e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:45.859327 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3ea42-e7d5-405a-9ca6-2f90ff85cc0e None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:45.860693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 705/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 12:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:45.866107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecddc5a-7d56-4a0d-9b8d-4485e0ae50e1 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:45.866561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecddc5a-7d56-4a0d-9b8d-4485e0ae50e1 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:45.866666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecddc5a-7d56-4a0d-9b8d-4485e0ae50e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:45.944621 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-fecddc5a-7d56-4a0d-9b8d-4485e0ae50e1 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:45.945833 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-fecddc5a-7d56-4a0d-9b8d-4485e0ae50e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:45.947885 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:46.039750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 734/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 12:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:46.191082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3ea42-e7d5-405a-9ca6-2f90ff85cc0e None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:46.192529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e6a3ea42-e7d5-405a-9ca6-2f90ff85cc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:46.336873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3ea42-e7d5-405a-9ca6-2f90ff85cc0e None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:46.362809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:46.447517 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3041bd5-1123-4643-a243-4aed410c4f23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:46.448055 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3041bd5-1123-4643-a243-4aed410c4f23 service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:46.448330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3041bd5-1123-4643-a243-4aed410c4f23 service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:46.448601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3041bd5-1123-4643-a243-4aed410c4f23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:46.450245 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a3041bd5-1123-4643-a243-4aed410c4f23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:46.537328 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3041bd5-1123-4643-a243-4aed410c4f23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:46.538906 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3041bd5-1123-4643-a243-4aed410c4f23 service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:46.556661 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 706/1440] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:48.438953 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb892f0-d69d-442d-b376-52e697bf3010 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:48.439520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb892f0-d69d-442d-b376-52e697bf3010 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:48.439520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb892f0-d69d-442d-b376-52e697bf3010 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:48.440830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 735/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 12:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:48.446447 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc3dda-6a76-4a1a-830a-0af57b4ca646 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:48.446900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc3dda-6a76-4a1a-830a-0af57b4ca646 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:48.447061 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc3dda-6a76-4a1a-830a-0af57b4ca646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:48.487444 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:48.513106 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-23fc3dda-6a76-4a1a-830a-0af57b4ca646 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:48.514312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-23fc3dda-6a76-4a1a-830a-0af57b4ca646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:48.609450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 707/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 12:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:48.649094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb892f0-d69d-442d-b376-52e697bf3010 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:48.650659 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-9eb892f0-d69d-442d-b376-52e697bf3010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:48.751666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb892f0-d69d-442d-b376-52e697bf3010 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:48.783049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:48.823956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c558a-39ef-4bc2-9cfb-74d326a8c543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:48.824570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c558a-39ef-4bc2-9cfb-74d326a8c543 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:48.824848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c558a-39ef-4bc2-9cfb-74d326a8c543 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:48.825119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c558a-39ef-4bc2-9cfb-74d326a8c543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:48.826776 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d29c558a-39ef-4bc2-9cfb-74d326a8c543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:48.932480 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c558a-39ef-4bc2-9cfb-74d326a8c543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:48.933903 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c558a-39ef-4bc2-9cfb-74d326a8c543 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:48.952356 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 736/1443] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:51.059118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a552c-d06a-4579-997c-92dae75376e7 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:51.059639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a552c-d06a-4579-997c-92dae75376e7 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:51.059833 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a552c-d06a-4579-997c-92dae75376e7 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:51.061021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 708/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 12:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:51.066058 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16419b3b-3086-4489-9c85-5bba5044fd9c None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:51.066510 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16419b3b-3086-4489-9c85-5bba5044fd9c None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:51.066625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16419b3b-3086-4489-9c85-5bba5044fd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:51.131542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-16419b3b-3086-4489-9c85-5bba5044fd9c None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:51.132666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-16419b3b-3086-4489-9c85-5bba5044fd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:51.147199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:51.233838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 737/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 12:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:51.270908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a552c-d06a-4579-997c-92dae75376e7 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:51.272440 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-cc7a552c-d06a-4579-997c-92dae75376e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:51.376111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a552c-d06a-4579-997c-92dae75376e7 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:51.388949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:51.450222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664622b6-5935-4d85-9cd4-9796da3006ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:51.450871 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664622b6-5935-4d85-9cd4-9796da3006ad service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:51.451149 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664622b6-5935-4d85-9cd4-9796da3006ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:51.451417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664622b6-5935-4d85-9cd4-9796da3006ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:51.453025 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-664622b6-5935-4d85-9cd4-9796da3006ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:51.558233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664622b6-5935-4d85-9cd4-9796da3006ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:51.559681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664622b6-5935-4d85-9cd4-9796da3006ad service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:51.577581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 709/1446] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:53.885833 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f964b-8a47-4912-b5c5-e985ae5753bc None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:53.886308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f964b-8a47-4912-b5c5-e985ae5753bc None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:53.886407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f964b-8a47-4912-b5c5-e985ae5753bc None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:53.887783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 738/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 12:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:53.893306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a25236-314e-4374-8faa-223d9a08656e None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:53.893733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a25236-314e-4374-8faa-223d9a08656e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:53.893842 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a25236-314e-4374-8faa-223d9a08656e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:53.941383 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:53.959779 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-15a25236-314e-4374-8faa-223d9a08656e None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:53.960979 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-15a25236-314e-4374-8faa-223d9a08656e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:54.056685 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 710/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 12:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:54.095582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f964b-8a47-4912-b5c5-e985ae5753bc None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:54.097098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a22f964b-8a47-4912-b5c5-e985ae5753bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:54.198298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f964b-8a47-4912-b5c5-e985ae5753bc None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:54.236462 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:54.271420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2cd17-6673-440e-91c9-8aac8a885ece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:54.272029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2cd17-6673-440e-91c9-8aac8a885ece service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:54.272302 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2cd17-6673-440e-91c9-8aac8a885ece service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:54.272569 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2cd17-6673-440e-91c9-8aac8a885ece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:54.274164 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b2b2cd17-6673-440e-91c9-8aac8a885ece service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:54.383138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2cd17-6673-440e-91c9-8aac8a885ece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:54.384585 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2cd17-6673-440e-91c9-8aac8a885ece service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:54.403155 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 739/1449] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:46:56.513945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c0b2f-0b8a-4151-90cf-27ed1a0bb76c None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:56.514577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c0b2f-0b8a-4151-90cf-27ed1a0bb76c None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:56.514577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c0b2f-0b8a-4151-90cf-27ed1a0bb76c None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:56.515883 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 711/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 12:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:56.521621 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1fc94-8b84-4ae5-bfe5-4d1a12ce2c6f None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:56.522028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1fc94-8b84-4ae5-bfe5-4d1a12ce2c6f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:56.522174 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe1fc94-8b84-4ae5-bfe5-4d1a12ce2c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:56.588592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-5fe1fc94-8b84-4ae5-bfe5-4d1a12ce2c6f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:56.589801 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5fe1fc94-8b84-4ae5-bfe5-4d1a12ce2c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:56.610380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:56.688421 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 740/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 12:46:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:46:56.727730 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c0b2f-0b8a-4151-90cf-27ed1a0bb76c None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:56.729245 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-786c0b2f-0b8a-4151-90cf-27ed1a0bb76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:56.829599 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c0b2f-0b8a-4151-90cf-27ed1a0bb76c None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:56.849330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:56.904697 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ca0eea-5b8e-470a-a3f1-ef05c1db19bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:56.905308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca0eea-5b8e-470a-a3f1-ef05c1db19bc service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:56.905582 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca0eea-5b8e-470a-a3f1-ef05c1db19bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:56.905849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ca0eea-5b8e-470a-a3f1-ef05c1db19bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:56.907520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a1ca0eea-5b8e-470a-a3f1-ef05c1db19bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:57.013875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca0eea-5b8e-470a-a3f1-ef05c1db19bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:57.015330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ca0eea-5b8e-470a-a3f1-ef05c1db19bc service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:57.033046 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 712/1452] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:46:59.081278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c79a6c-b89b-4e10-a8a1-46c06eca88f6 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:59.081802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c79a6c-b89b-4e10-a8a1-46c06eca88f6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:59.081902 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c79a6c-b89b-4e10-a8a1-46c06eca88f6 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:59.083243 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 741/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 12:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:46:59.088405 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f02709-e588-4aad-8363-43298793835f None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:59.088830 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f02709-e588-4aad-8363-43298793835f None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:59.088955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f02709-e588-4aad-8363-43298793835f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:59.151184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:59.153795 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-f1f02709-e588-4aad-8363-43298793835f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:46:59.155585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f1f02709-e588-4aad-8363-43298793835f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:59.250669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 713/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 12:46:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:46:59.289230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c79a6c-b89b-4e10-a8a1-46c06eca88f6 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:46:59.290778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e8c79a6c-b89b-4e10-a8a1-46c06eca88f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:59.391316 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c79a6c-b89b-4e10-a8a1-46c06eca88f6 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:46:59.439900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:46:59.464504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af06430-4820-46e5-8454-c11d9a077ea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:46:59.465116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af06430-4820-46e5-8454-c11d9a077ea2 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:46:59.465391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af06430-4820-46e5-8454-c11d9a077ea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:46:59.465659 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af06430-4820-46e5-8454-c11d9a077ea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:46:59.467301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-7af06430-4820-46e5-8454-c11d9a077ea2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:46:59.572637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af06430-4820-46e5-8454-c11d9a077ea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:46:59.574037 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af06430-4820-46e5-8454-c11d9a077ea2 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:46:59.593601 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 742/1455] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:46:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:47:01.637983 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db8a13-ee62-41f0-b327-e3e50996c0cf None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:01.638580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db8a13-ee62-41f0-b327-e3e50996c0cf None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:01.638580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db8a13-ee62-41f0-b327-e3e50996c0cf None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:01.639915 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 714/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 12:47:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:47:01.645727 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e0bca-8991-448a-b2c4-e9bbf5715996 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:01.646171 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e0bca-8991-448a-b2c4-e9bbf5715996 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:01.646278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e0bca-8991-448a-b2c4-e9bbf5715996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:01.711711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1e7e0bca-8991-448a-b2c4-e9bbf5715996 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:47:01.712907 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1e7e0bca-8991-448a-b2c4-e9bbf5715996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:01.742637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:01.810031 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 743/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 12:47:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:47:01.847675 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db8a13-ee62-41f0-b327-e3e50996c0cf None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:47:01.849873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-b7db8a13-ee62-41f0-b327-e3e50996c0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:01.952407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db8a13-ee62-41f0-b327-e3e50996c0cf None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:47:01.980213 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:02.028298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e2816-7427-4c89-a6cc-e08c08f091c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:47:02.028909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e2816-7427-4c89-a6cc-e08c08f091c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:02.029188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e2816-7427-4c89-a6cc-e08c08f091c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:02.029457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e2816-7427-4c89-a6cc-e08c08f091c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:02.031098 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ee3e2816-7427-4c89-a6cc-e08c08f091c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:02.136645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e2816-7427-4c89-a6cc-e08c08f091c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:47:02.138069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e2816-7427-4c89-a6cc-e08c08f091c1 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:47:02.156156 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 715/1458] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:47:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:47:04.197154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fa7d7-837a-435a-a7aa-24ef96245311 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:04.197669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fa7d7-837a-435a-a7aa-24ef96245311 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:04.197669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fa7d7-837a-435a-a7aa-24ef96245311 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:04.199064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 744/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 12:47:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:47:04.204359 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f7781-7e6d-4e71-b338-23b3012c9e1a None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:04.204791 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f7781-7e6d-4e71-b338-23b3012c9e1a None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:04.204897 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f7781-7e6d-4e71-b338-23b3012c9e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:04.269591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a56f7781-7e6d-4e71-b338-23b3012c9e1a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:47:04.270860 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a56f7781-7e6d-4e71-b338-23b3012c9e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:04.278996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:04.367256 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 716/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 12:47:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:47:04.404991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fa7d7-837a-435a-a7aa-24ef96245311 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:47:04.406582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-570fa7d7-837a-435a-a7aa-24ef96245311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:04.509148 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fa7d7-837a-435a-a7aa-24ef96245311 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:47:04.563665 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:04.582022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1f2b8-d923-4f73-b7de-efb31f64d906 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:47:04.583182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1f2b8-d923-4f73-b7de-efb31f64d906 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:04.583467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1f2b8-d923-4f73-b7de-efb31f64d906 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:04.583741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1f2b8-d923-4f73-b7de-efb31f64d906 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:04.585405 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e0c1f2b8-d923-4f73-b7de-efb31f64d906 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:04.691867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1f2b8-d923-4f73-b7de-efb31f64d906 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:47:04.693283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1f2b8-d923-4f73-b7de-efb31f64d906 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:47:04.711773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 745/1461] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:47:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:47:06.738148 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0587e6-e548-4152-b897-24966c45f3b3 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:06.739241 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0587e6-e548-4152-b897-24966c45f3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:06.739241 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0587e6-e548-4152-b897-24966c45f3b3 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:06.740562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 717/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 12:47:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:47:06.745656 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026d597-d80e-4701-bbae-9f9489ab1a87 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:06.746056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026d597-d80e-4701-bbae-9f9489ab1a87 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:06.746205 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026d597-d80e-4701-bbae-9f9489ab1a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:06.810770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-b026d597-d80e-4701-bbae-9f9489ab1a87 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:47:06.811976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b026d597-d80e-4701-bbae-9f9489ab1a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:06.849309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:06.908329 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 746/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 12:47:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:47:06.946671 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0587e6-e548-4152-b897-24966c45f3b3 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:47:06.948221 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-fc0587e6-e548-4152-b897-24966c45f3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:07.050173 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0587e6-e548-4152-b897-24966c45f3b3 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:47:07.087488 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:07.123185 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3fa065-a855-449f-8d53-19fe1bcf746c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:47:07.123799 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fa065-a855-449f-8d53-19fe1bcf746c service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:07.124080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fa065-a855-449f-8d53-19fe1bcf746c service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:07.124353 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3fa065-a855-449f-8d53-19fe1bcf746c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:07.125977 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-7d3fa065-a855-449f-8d53-19fe1bcf746c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:07.232160 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3fa065-a855-449f-8d53-19fe1bcf746c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:47:07.233561 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3fa065-a855-449f-8d53-19fe1bcf746c service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:47:07.251617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 718/1464] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:47:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:47:09.256931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8df65-f124-4463-813e-eddd40852d53 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:09.257532 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8df65-f124-4463-813e-eddd40852d53 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:09.257532 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea8df65-f124-4463-813e-eddd40852d53 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:09.258857 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 747/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 12:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:47:09.263973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a00a74-0c5b-4383-9710-8de2a2a88f58 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:09.264430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a00a74-0c5b-4383-9710-8de2a2a88f58 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:09.264537 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a00a74-0c5b-4383-9710-8de2a2a88f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:09.329885 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-91a00a74-0c5b-4383-9710-8de2a2a88f58 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:47:09.331127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-91a00a74-0c5b-4383-9710-8de2a2a88f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:09.349317 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:09.428046 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 719/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 12:47:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:47:09.464318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8df65-f124-4463-813e-eddd40852d53 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:47:09.465831 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-aea8df65-f124-4463-813e-eddd40852d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:09.567821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea8df65-f124-4463-813e-eddd40852d53 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:47:09.633250 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:09.640716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532925c-e540-4d07-9450-4ece9a621baf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:47:09.641332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532925c-e540-4d07-9450-4ece9a621baf service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:09.641615 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532925c-e540-4d07-9450-4ece9a621baf service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:09.641890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532925c-e540-4d07-9450-4ece9a621baf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:09.643584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f532925c-e540-4d07-9450-4ece9a621baf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:09.750146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532925c-e540-4d07-9450-4ece9a621baf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:47:09.751729 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532925c-e540-4d07-9450-4ece9a621baf service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:47:09.770498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 748/1467] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:47:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:47:11.773875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ead36-9d49-4146-a8e7-448b6bd4a841 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:11.774443 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ead36-9d49-4146-a8e7-448b6bd4a841 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:11.774443 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ead36-9d49-4146-a8e7-448b6bd4a841 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:11.775818 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 720/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 12:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:47:11.781201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f2a51-5a90-4b43-81d8-79c5e0dc194b None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:11.781593 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f2a51-5a90-4b43-81d8-79c5e0dc194b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:11.781760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f2a51-5a90-4b43-81d8-79c5e0dc194b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:11.846466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-2c1f2a51-5a90-4b43-81d8-79c5e0dc194b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:47:11.847693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-2c1f2a51-5a90-4b43-81d8-79c5e0dc194b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:11.896362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:11.945539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 749/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 12:47:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:47:11.983429 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ead36-9d49-4146-a8e7-448b6bd4a841 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:47:11.985030 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-be2ead36-9d49-4146-a8e7-448b6bd4a841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:12.088193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ead36-9d49-4146-a8e7-448b6bd4a841 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:47:12.134586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:12.160111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1a62b-53a7-4750-aa8f-6df6f54f7865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:47:12.160754 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1a62b-53a7-4750-aa8f-6df6f54f7865 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:12.161035 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1a62b-53a7-4750-aa8f-6df6f54f7865 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:12.161305 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1a62b-53a7-4750-aa8f-6df6f54f7865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:12.163021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-18d1a62b-53a7-4750-aa8f-6df6f54f7865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:12.272455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1a62b-53a7-4750-aa8f-6df6f54f7865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:47:12.273877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1a62b-53a7-4750-aa8f-6df6f54f7865 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:47:12.291593 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 721/1470] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:47:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:47:21.943732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=22 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:22.289196 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=66 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:31.598560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bedb63-eec2-48a9-8812-d615be35b87f None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:31.599070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bedb63-eec2-48a9-8812-d615be35b87f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:31.599070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bedb63-eec2-48a9-8812-d615be35b87f None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:31.600420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 750/1471] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:47:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:47:31.605480 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d1cd8-41a8-4243-86ff-3b93c3db7690 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:31.605903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d1cd8-41a8-4243-86ff-3b93c3db7690 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:31.606025 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016d1cd8-41a8-4243-86ff-3b93c3db7690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:31.672097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-016d1cd8-41a8-4243-86ff-3b93c3db7690 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:47:31.673280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-016d1cd8-41a8-4243-86ff-3b93c3db7690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:31.766388 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 722/1472] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:47:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:47:31.849872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fccef9-f6fd-4e94-9530-35f1a53c96de None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:31.850107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fccef9-f6fd-4e94-9530-35f1a53c96de None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:31.850378 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fccef9-f6fd-4e94-9530-35f1a53c96de None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:31.851746 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 751/1473] 10.222.0.65 () {54 vars in 966 bytes} [Wed Aug 3 12:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:47:31.856678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd6f5-1751-4b92-b55c-4d898601adfe None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:31.856995 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd6f5-1751-4b92-b55c-4d898601adfe None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:31.857318 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd6f5-1751-4b92-b55c-4d898601adfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:31.932383 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-956bd6f5-1751-4b92-b55c-4d898601adfe None None] MFA Rules not processed for user `e6066e48aa644a47a889e09cfe204858`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:47:31.933621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-956bd6f5-1751-4b92-b55c-4d898601adfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:31.956709 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:32.015744 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 723/1474] 10.222.0.65 () {58 vars in 1063 bytes} [Wed Aug 3 12:47:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:47:32.024383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fccef9-f6fd-4e94-9530-35f1a53c96de None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:47:32.025890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b2fccef9-f6fd-4e94-9530-35f1a53c96de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:32.127618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fccef9-f6fd-4e94-9530-35f1a53c96de None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:47:32.198791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a18233-3b24-4d50-b967-355e4e5e4b1f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6066e48aa644a47a889e09cfe204858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:47:32.199399 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a18233-3b24-4d50-b967-355e4e5e4b1f service glance] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:32.199695 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a18233-3b24-4d50-b967-355e4e5e4b1f service glance] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:32.199951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a18233-3b24-4d50-b967-355e4e5e4b1f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:32.201561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-51a18233-3b24-4d50-b967-355e4e5e4b1f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:32.255309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:32.313038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a18233-3b24-4d50-b967-355e4e5e4b1f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:47:32.314454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a18233-3b24-4d50-b967-355e4e5e4b1f service glance] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:47:32.333109 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 752/1475] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:47:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:47:34.394058 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e137d0-9d63-440c-bc05-4718a6389c03 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:34.394615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e137d0-9d63-440c-bc05-4718a6389c03 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:34.394785 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e137d0-9d63-440c-bc05-4718a6389c03 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:34.396081 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 724/1476] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:47:34.401840 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888918c-1b8f-45ec-b9d4-208436bf90e3 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:34.402230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888918c-1b8f-45ec-b9d4-208436bf90e3 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:34.402394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888918c-1b8f-45ec-b9d4-208436bf90e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:34.467154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-f888918c-1b8f-45ec-b9d4-208436bf90e3 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:47:34.468353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f888918c-1b8f-45ec-b9d4-208436bf90e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:34.562712 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 753/1477] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:47:34.586174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e137d0-9d63-440c-bc05-4718a6389c03 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:47:34.587751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-27e137d0-9d63-440c-bc05-4718a6389c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:34.712208 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e137d0-9d63-440c-bc05-4718a6389c03 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:47:34.760521 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:34.804071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46f9f3-44ad-46ec-bc6e-75fd034b6cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:47:34.804675 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46f9f3-44ad-46ec-bc6e-75fd034b6cbe admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:34.804950 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46f9f3-44ad-46ec-bc6e-75fd034b6cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:34.805221 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46f9f3-44ad-46ec-bc6e-75fd034b6cbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:34.805900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46f9f3-44ad-46ec-bc6e-75fd034b6cbe admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:47:34.807673 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: WARNING py.warnings [None req-ef46f9f3-44ad-46ec-bc6e-75fd034b6cbe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 12:47:34.807673 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: warnings.warn(msg) Aug 03 12:47:34.807673 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: Aug 03 12:47:34.808826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46f9f3-44ad-46ec-bc6e-75fd034b6cbe admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:47:34.827253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 725/1478] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 3 12:47:34 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:47:36.828433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911a911-3c27-4b26-8753-dbe46b3bfb6c None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:36.829016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911a911-3c27-4b26-8753-dbe46b3bfb6c None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:36.829016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911a911-3c27-4b26-8753-dbe46b3bfb6c None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:36.830388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 754/1479] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:47:36.835982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55abefb-493e-4563-969c-6894a985475e None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:36.836409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55abefb-493e-4563-969c-6894a985475e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:36.836574 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55abefb-493e-4563-969c-6894a985475e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:36.902179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-e55abefb-493e-4563-969c-6894a985475e None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:47:36.903433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e55abefb-493e-4563-969c-6894a985475e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:36.997693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 726/1480] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:47:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:47:37.040712 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911a911-3c27-4b26-8753-dbe46b3bfb6c None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:47:37.042453 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-7911a911-3c27-4b26-8753-dbe46b3bfb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:37.052431 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:37.186262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911a911-3c27-4b26-8753-dbe46b3bfb6c None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:47:37.263685 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:37.296600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86bb6f3-ef0c-4c7a-a6bc-0ba809615c71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:47:37.297212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bb6f3-ef0c-4c7a-a6bc-0ba809615c71 service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:37.297489 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bb6f3-ef0c-4c7a-a6bc-0ba809615c71 service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:37.297757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86bb6f3-ef0c-4c7a-a6bc-0ba809615c71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:37.299399 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e86bb6f3-ef0c-4c7a-a6bc-0ba809615c71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:37.386262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86bb6f3-ef0c-4c7a-a6bc-0ba809615c71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:47:37.387887 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86bb6f3-ef0c-4c7a-a6bc-0ba809615c71 service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:47:37.406052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 755/1481] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:47:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:47:39.604749 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c65606-40cf-4083-b78b-4cb8e5ea79e8 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:39.605321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c65606-40cf-4083-b78b-4cb8e5ea79e8 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:39.605321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c65606-40cf-4083-b78b-4cb8e5ea79e8 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:39.606689 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 727/1482] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:47:39.611866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e2b2f-5d99-4c3e-9880-5dee57562556 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:39.612277 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e2b2f-5d99-4c3e-9880-5dee57562556 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:39.612532 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e2b2f-5d99-4c3e-9880-5dee57562556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:39.681129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-e99e2b2f-5d99-4c3e-9880-5dee57562556 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:47:39.682324 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e99e2b2f-5d99-4c3e-9880-5dee57562556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:39.728289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:39.778193 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 756/1483] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:47:39.814440 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c65606-40cf-4083-b78b-4cb8e5ea79e8 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:47:39.816048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-86c65606-40cf-4083-b78b-4cb8e5ea79e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:39.839849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:39.919482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c65606-40cf-4083-b78b-4cb8e5ea79e8 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:47:39.994062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eabe2cc-7b38-4572-8b4b-39d60ea65513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:47:39.994708 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabe2cc-7b38-4572-8b4b-39d60ea65513 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:39.994990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabe2cc-7b38-4572-8b4b-39d60ea65513 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:39.995281 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eabe2cc-7b38-4572-8b4b-39d60ea65513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:39.996881 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-0eabe2cc-7b38-4572-8b4b-39d60ea65513 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:40.070898 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:40.107815 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabe2cc-7b38-4572-8b4b-39d60ea65513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:47:40.109239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eabe2cc-7b38-4572-8b4b-39d60ea65513 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:47:40.128276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 728/1484] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:47:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:47:42.184303 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b6573-46fb-4e7e-bf1b-cb15367a7da8 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:42.184954 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b6573-46fb-4e7e-bf1b-cb15367a7da8 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:42.184954 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b6573-46fb-4e7e-bf1b-cb15367a7da8 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:42.186306 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 757/1485] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:47:42.192733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde8ba5-52e9-495d-ad4b-7c774f1301a6 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:42.193246 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde8ba5-52e9-495d-ad4b-7c774f1301a6 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:42.193371 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde8ba5-52e9-495d-ad4b-7c774f1301a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:42.265711 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-5dde8ba5-52e9-495d-ad4b-7c774f1301a6 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:47:42.266963 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-5dde8ba5-52e9-495d-ad4b-7c774f1301a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:42.364164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 729/1486] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:47:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:47:42.401957 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b6573-46fb-4e7e-bf1b-cb15367a7da8 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:47:42.403586 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-627b6573-46fb-4e7e-bf1b-cb15367a7da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:42.515541 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b6573-46fb-4e7e-bf1b-cb15367a7da8 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:47:42.601473 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:42.603464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55e66b-e331-457c-80c6-992c67c9d7f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:47:42.604121 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55e66b-e331-457c-80c6-992c67c9d7f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:42.604580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55e66b-e331-457c-80c6-992c67c9d7f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:42.605007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55e66b-e331-457c-80c6-992c67c9d7f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:42.606937 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ad55e66b-e331-457c-80c6-992c67c9d7f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:42.714354 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55e66b-e331-457c-80c6-992c67c9d7f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:47:42.715829 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55e66b-e331-457c-80c6-992c67c9d7f6 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:47:42.734272 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 758/1487] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:47:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:47:44.820475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c82e46-024a-4426-947e-138b824bcab8 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:44.821058 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c82e46-024a-4426-947e-138b824bcab8 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:44.821058 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c82e46-024a-4426-947e-138b824bcab8 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:44.822377 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 730/1488] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:47:44.827544 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df45c0-c0e7-4390-b6f4-a2ec6401a3f7 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:44.827986 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df45c0-c0e7-4390-b6f4-a2ec6401a3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:44.828095 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df45c0-c0e7-4390-b6f4-a2ec6401a3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:44.893598 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-f8df45c0-c0e7-4390-b6f4-a2ec6401a3f7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:47:44.894842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f8df45c0-c0e7-4390-b6f4-a2ec6401a3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:44.947401 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:44.992483 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 759/1489] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:47:45.032257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c82e46-024a-4426-947e-138b824bcab8 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:47:45.033819 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-26c82e46-024a-4426-947e-138b824bcab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:45.069389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:45.136617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c82e46-024a-4426-947e-138b824bcab8 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:47:45.208296 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d31cca-8d04-45e7-a9b1-84324f20675a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:47:45.208902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d31cca-8d04-45e7-a9b1-84324f20675a service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:45.209175 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d31cca-8d04-45e7-a9b1-84324f20675a service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:45.209444 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d31cca-8d04-45e7-a9b1-84324f20675a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:45.211070 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-60d31cca-8d04-45e7-a9b1-84324f20675a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:45.289277 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:45.317885 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d31cca-8d04-45e7-a9b1-84324f20675a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:47:45.319311 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d31cca-8d04-45e7-a9b1-84324f20675a service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:47:45.338978 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 731/1490] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:47:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:47:47.490449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b62f2f-cf6c-4275-9579-2e873e64559c None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:47.491040 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b62f2f-cf6c-4275-9579-2e873e64559c None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:47.491040 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b62f2f-cf6c-4275-9579-2e873e64559c None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:47.492367 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 760/1491] 10.222.0.65 () {54 vars in 926 bytes} [Wed Aug 3 12:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:47:47.497603 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d97086-10c9-43ef-8a46-6c223cb4fe57 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:47.498007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d97086-10c9-43ef-8a46-6c223cb4fe57 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:47.498152 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d97086-10c9-43ef-8a46-6c223cb4fe57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:47.562637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-65d97086-10c9-43ef-8a46-6c223cb4fe57 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:47:47.563834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-65d97086-10c9-43ef-8a46-6c223cb4fe57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:47.667055 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 732/1492] 10.222.0.65 () {58 vars in 1023 bytes} [Wed Aug 3 12:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:47:47.706041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b62f2f-cf6c-4275-9579-2e873e64559c None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:47:47.707607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-98b62f2f-cf6c-4275-9579-2e873e64559c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:47.810819 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b62f2f-cf6c-4275-9579-2e873e64559c None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:47:47.882396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dfca48-0514-48c0-8d34-76eb3346fdb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:47:47.883041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfca48-0514-48c0-8d34-76eb3346fdb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:47:47.883316 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfca48-0514-48c0-8d34-76eb3346fdb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:47:47.883588 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfca48-0514-48c0-8d34-76eb3346fdb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:47:47.885186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-00dfca48-0514-48c0-8d34-76eb3346fdb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:47:47.895395 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:48.002534 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfca48-0514-48c0-8d34-76eb3346fdb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:47:48.003994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dfca48-0514-48c0-8d34-76eb3346fdb7 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:47:48.022058 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 761/1493] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:47:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:47:57.665242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=82 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:47:58.019247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=50 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:49:14.135808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7629a2e-e600-4dd4-b7e6-eaa7bf876f3f None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:14.136376 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7629a2e-e600-4dd4-b7e6-eaa7bf876f3f None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:14.136376 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7629a2e-e600-4dd4-b7e6-eaa7bf876f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:14.210172 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a7629a2e-e600-4dd4-b7e6-eaa7bf876f3f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:49:14.211477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a7629a2e-e600-4dd4-b7e6-eaa7bf876f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:49:14.308768 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 733/1494] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:49:14.316907 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0efd3d-66bf-497d-956f-915b79a58b1c None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:14.317404 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0efd3d-66bf-497d-956f-915b79a58b1c None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:14.317511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0efd3d-66bf-497d-956f-915b79a58b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:14.391619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-0a0efd3d-66bf-497d-956f-915b79a58b1c None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:49:14.392763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0a0efd3d-66bf-497d-956f-915b79a58b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:49:14.493794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 762/1495] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:49:14.498598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7629a2e-e600-4dd4-b7e6-eaa7bf876f3f None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:49:14.499795 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a7629a2e-e600-4dd4-b7e6-eaa7bf876f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:49:14.511012 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7629a2e-e600-4dd4-b7e6-eaa7bf876f3f None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:49:14.572881 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:49:14.607018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec076391-c0e7-4241-a842-3648aa3d1826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:49:14.607632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec076391-c0e7-4241-a842-3648aa3d1826 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:14.607908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec076391-c0e7-4241-a842-3648aa3d1826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:14.608178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec076391-c0e7-4241-a842-3648aa3d1826 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:14.608865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec076391-c0e7-4241-a842-3648aa3d1826 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:49:14.609165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec076391-c0e7-4241-a842-3648aa3d1826 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:49:14.610167 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: WARNING py.warnings [None req-ec076391-c0e7-4241-a842-3648aa3d1826 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 03 12:49:14.610167 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: warnings.warn(msg) Aug 03 12:49:14.610167 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: Aug 03 12:49:14.611070 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec076391-c0e7-4241-a842-3648aa3d1826 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:49:14.622006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 734/1496] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:49:14.630959 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0efd3d-66bf-497d-956f-915b79a58b1c None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:49:14.640224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0efd3d-66bf-497d-956f-915b79a58b1c None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:49:14.701241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:49:14.734686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50600f5a-b47a-4c5b-b1a6-d65d217148ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:49:14.735296 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50600f5a-b47a-4c5b-b1a6-d65d217148ed admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:14.735573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50600f5a-b47a-4c5b-b1a6-d65d217148ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:14.735843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50600f5a-b47a-4c5b-b1a6-d65d217148ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:14.736508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50600f5a-b47a-4c5b-b1a6-d65d217148ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:49:14.738199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: WARNING py.warnings [None req-50600f5a-b47a-4c5b-b1a6-d65d217148ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 12:49:14.738199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: warnings.warn(msg) Aug 03 12:49:14.738199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: Aug 03 12:49:14.739422 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50600f5a-b47a-4c5b-b1a6-d65d217148ed admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:49:14.793415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 763/1497] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:49:14 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:49:14.797654 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec076391-c0e7-4241-a842-3648aa3d1826 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:49:14.799233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ec076391-c0e7-4241-a842-3648aa3d1826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:49:14.834466 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec076391-c0e7-4241-a842-3648aa3d1826 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:49:14.931409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5fd8d-4091-455f-a88e-78feacb6bf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:49:14.932025 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5fd8d-4091-455f-a88e-78feacb6bf16 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:14.932304 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5fd8d-4091-455f-a88e-78feacb6bf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:14.932571 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5fd8d-4091-455f-a88e-78feacb6bf16 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:14.933222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5fd8d-4091-455f-a88e-78feacb6bf16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:49:14.934226 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: WARNING py.warnings [None req-1ce5fd8d-4091-455f-a88e-78feacb6bf16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 12:49:14.934226 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: warnings.warn(msg) Aug 03 12:49:14.934226 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: Aug 03 12:49:14.935453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5fd8d-4091-455f-a88e-78feacb6bf16 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:49:14.976713 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 735/1498] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:49:14 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:49:14.980766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50600f5a-b47a-4c5b-b1a6-d65d217148ed admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:49:14.982260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-50600f5a-b47a-4c5b-b1a6-d65d217148ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:49:15.043085 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50600f5a-b47a-4c5b-b1a6-d65d217148ed admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:49:15.125394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:49:15.140130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac985833-7e9e-4c5b-95b8-298a861a115d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:49:15.140752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac985833-7e9e-4c5b-95b8-298a861a115d admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:15.141035 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac985833-7e9e-4c5b-95b8-298a861a115d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:15.141306 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac985833-7e9e-4c5b-95b8-298a861a115d admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:15.142004 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac985833-7e9e-4c5b-95b8-298a861a115d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:49:15.143063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: WARNING py.warnings [None req-ac985833-7e9e-4c5b-95b8-298a861a115d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 03 12:49:15.143063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: warnings.warn(msg) Aug 03 12:49:15.143063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: Aug 03 12:49:15.143951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac985833-7e9e-4c5b-95b8-298a861a115d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:49:15.155998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 764/1499] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:49:15.162288 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5fd8d-4091-455f-a88e-78feacb6bf16 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:49:15.163825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1ce5fd8d-4091-455f-a88e-78feacb6bf16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:49:15.175843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5fd8d-4091-455f-a88e-78feacb6bf16 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:49:15.201630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:49:15.269578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fefac3-8409-44e9-bec8-e6feee8f1223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:49:15.270116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fefac3-8409-44e9-bec8-e6feee8f1223 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:15.270394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fefac3-8409-44e9-bec8-e6feee8f1223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:15.270690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fefac3-8409-44e9-bec8-e6feee8f1223 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:15.271383 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fefac3-8409-44e9-bec8-e6feee8f1223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:49:15.272392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: WARNING py.warnings [None req-93fefac3-8409-44e9-bec8-e6feee8f1223 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 03 12:49:15.272392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: warnings.warn(msg) Aug 03 12:49:15.272392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: Aug 03 12:49:15.273271 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fefac3-8409-44e9-bec8-e6feee8f1223 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:49:15.285225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 736/1500] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:49:15 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:49:15.289415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac985833-7e9e-4c5b-95b8-298a861a115d admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:49:15.290963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ac985833-7e9e-4c5b-95b8-298a861a115d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:49:15.324660 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac985833-7e9e-4c5b-95b8-298a861a115d admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:49:15.416218 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11c554-9968-40c6-9467-c65732cf0550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:49:15.416939 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11c554-9968-40c6-9467-c65732cf0550 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:15.417223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11c554-9968-40c6-9467-c65732cf0550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:15.417506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11c554-9968-40c6-9467-c65732cf0550 admin admin] PATH_INFO: `/v3/projects/cbbaee8a85c9455e9a31e0220990132f/users/7a217b2f86de4876a77d93a446a4431e/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:15.440168 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11c554-9968-40c6-9467-c65732cf0550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbbaee8a85c9455e9a31e0220990132f, user_id=7a217b2f86de4876a77d93a446a4431e, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:49:15.441186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: WARNING py.warnings [None req-3d11c554-9968-40c6-9467-c65732cf0550 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 12:49:15.441186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: warnings.warn(msg) Aug 03 12:49:15.441186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: Aug 03 12:49:15.443158 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11c554-9968-40c6-9467-c65732cf0550 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:49:15.466689 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 765/1501] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:49:15 2022] PUT /identity/v3/projects/cbbaee8a85c9455e9a31e0220990132f/users/7a217b2f86de4876a77d93a446a4431e/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:49:15.479291 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bdc87c-718b-4e61-9634-5a5b814f4c97 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:15.479585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bdc87c-718b-4e61-9634-5a5b814f4c97 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:15.479866 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bdc87c-718b-4e61-9634-5a5b814f4c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:15.535417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-c8bdc87c-718b-4e61-9634-5a5b814f4c97 None None] MFA Rules not processed for user `7a217b2f86de4876a77d93a446a4431e`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:49:15.536541 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c8bdc87c-718b-4e61-9634-5a5b814f4c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:49:15.612978 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 737/1502] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:49:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:49:15.626162 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244e3ed-cc3d-4bc7-bb6c-300414e7375a None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:15.626445 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244e3ed-cc3d-4bc7-bb6c-300414e7375a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:15.626748 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244e3ed-cc3d-4bc7-bb6c-300414e7375a None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:15.628109 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 766/1503] 10.222.0.65 () {54 vars in 964 bytes} [Wed Aug 3 12:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:49:15.632896 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e3302-b54e-440b-acfa-81b9791f3137 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:15.633183 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e3302-b54e-440b-acfa-81b9791f3137 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:15.633459 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63e3302-b54e-440b-acfa-81b9791f3137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:15.664320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:49:15.708809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-e63e3302-b54e-440b-acfa-81b9791f3137 None None] MFA Rules not processed for user `473a5e7e980b4473acec85905fa7078c`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:49:15.709989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e63e3302-b54e-440b-acfa-81b9791f3137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:49:15.785236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 738/1504] 10.222.0.65 () {58 vars in 1061 bytes} [Wed Aug 3 12:49:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:49:15.792976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244e3ed-cc3d-4bc7-bb6c-300414e7375a None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:49:15.794412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-6244e3ed-cc3d-4bc7-bb6c-300414e7375a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:49:15.843779 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:49:15.898142 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244e3ed-cc3d-4bc7-bb6c-300414e7375a None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:49:15.970396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b082a69-e940-4718-97f6-f7a1834f2558 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:49:15.971054 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b082a69-e940-4718-97f6-f7a1834f2558 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:15.971879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b082a69-e940-4718-97f6-f7a1834f2558 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:15.972181 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b082a69-e940-4718-97f6-f7a1834f2558 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:15.973827 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8b082a69-e940-4718-97f6-f7a1834f2558 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:49:16.069542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b082a69-e940-4718-97f6-f7a1834f2558 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:49:16.070929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b082a69-e940-4718-97f6-f7a1834f2558 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:49:16.074332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:49:16.088773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 767/1505] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:49:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:49:16.133134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017465b-7fa4-460e-becf-8a63c089c232 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:16.133422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017465b-7fa4-460e-becf-8a63c089c232 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:16.133698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017465b-7fa4-460e-becf-8a63c089c232 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:16.134982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 739/1506] 10.222.0.65 () {52 vars in 830 bytes} [Wed Aug 3 12:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:49:16.138377 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b082a69-e940-4718-97f6-f7a1834f2558 service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:49:16.139600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8b082a69-e940-4718-97f6-f7a1834f2558 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:49:16.236801 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b082a69-e940-4718-97f6-f7a1834f2558 service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:49:16.344640 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c7921-70ff-4c58-bf43-3dec44892faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:49:16.345276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c7921-70ff-4c58-bf43-3dec44892faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:16.345457 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c7921-70ff-4c58-bf43-3dec44892faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:16.345723 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c7921-70ff-4c58-bf43-3dec44892faa admin admin] PATH_INFO: `/v3/users/7a217b2f86de4876a77d93a446a4431e` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:16.357635 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c7921-70ff-4c58-bf43-3dec44892faa admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a217b2f86de4876a77d93a446a4431e)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:49:16.358669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: WARNING py.warnings [None req-bf6c7921-70ff-4c58-bf43-3dec44892faa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 12:49:16.358669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: warnings.warn(msg) Aug 03 12:49:16.358669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: Aug 03 12:49:16.359861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c7921-70ff-4c58-bf43-3dec44892faa admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:49:16.369260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:49:16.424396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bf6c7921-70ff-4c58-bf43-3dec44892faa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a217b2f86de4876a77d93a446a4431e'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:49:16.438676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bf6c7921-70ff-4c58-bf43-3dec44892faa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a217b2f86de4876a77d93a446a4431e'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:49:16.461552 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bf6c7921-70ff-4c58-bf43-3dec44892faa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a217b2f86de4876a77d93a446a4431e'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:49:16.462000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bf6c7921-70ff-4c58-bf43-3dec44892faa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a217b2f86de4876a77d93a446a4431e'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:49:16.462543 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bf6c7921-70ff-4c58-bf43-3dec44892faa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a217b2f86de4876a77d93a446a4431e'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:49:16.477741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 768/1507] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:49:16 2022] DELETE /identity/v3/users/7a217b2f86de4876a77d93a446a4431e => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:49:16.482166 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbef7f8-ec7b-40fe-bb1b-c6743d0ed7ae None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:16.482664 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbef7f8-ec7b-40fe-bb1b-c6743d0ed7ae None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:16.482769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbef7f8-ec7b-40fe-bb1b-c6743d0ed7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:16.540048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-2cbef7f8-ec7b-40fe-bb1b-c6743d0ed7ae None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:49:16.541625 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2cbef7f8-ec7b-40fe-bb1b-c6743d0ed7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:49:16.616613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:49:16.630536 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 740/1508] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:49:16.639850 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c7921-70ff-4c58-bf43-3dec44892faa admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:49:16.641088 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-bf6c7921-70ff-4c58-bf43-3dec44892faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:49:16.739207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c7921-70ff-4c58-bf43-3dec44892faa admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:49:16.793310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:49:16.814729 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85cfcd-3b58-49de-96a1-b3563739c196 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:49:16.815891 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85cfcd-3b58-49de-96a1-b3563739c196 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:16.816161 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85cfcd-3b58-49de-96a1-b3563739c196 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:16.816427 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85cfcd-3b58-49de-96a1-b3563739c196 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:16.818026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-bf85cfcd-3b58-49de-96a1-b3563739c196 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:49:16.929286 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85cfcd-3b58-49de-96a1-b3563739c196 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:49:16.930762 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85cfcd-3b58-49de-96a1-b3563739c196 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:49:16.948644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 769/1509] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:49:17.274542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbef7f8-ec7b-40fe-bb1b-c6743d0ed7ae None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:49:17.275859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2cbef7f8-ec7b-40fe-bb1b-c6743d0ed7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:49:17.288119 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbef7f8-ec7b-40fe-bb1b-c6743d0ed7ae None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:49:17.383254 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d33f72-ac35-411b-b7d0-69d5b44ce7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:49:17.383893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d33f72-ac35-411b-b7d0-69d5b44ce7de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:49:17.384177 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d33f72-ac35-411b-b7d0-69d5b44ce7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:49:17.384450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d33f72-ac35-411b-b7d0-69d5b44ce7de admin admin] PATH_INFO: `/v3/projects/cbbaee8a85c9455e9a31e0220990132f` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:49:17.394690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d33f72-ac35-411b-b7d0-69d5b44ce7de admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbbaee8a85c9455e9a31e0220990132f)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:49:17.395717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: WARNING py.warnings [None req-41d33f72-ac35-411b-b7d0-69d5b44ce7de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 12:49:17.395717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: warnings.warn(msg) Aug 03 12:49:17.395717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: Aug 03 12:49:17.396901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d33f72-ac35-411b-b7d0-69d5b44ce7de admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:49:17.454265 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-41d33f72-ac35-411b-b7d0-69d5b44ce7de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbbaee8a85c9455e9a31e0220990132f', 'request_id': 'req-41d33f72-ac35-411b-b7d0-69d5b44ce7de'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:49:17.463723 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-41d33f72-ac35-411b-b7d0-69d5b44ce7de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbbaee8a85c9455e9a31e0220990132f', 'request_id': 'req-41d33f72-ac35-411b-b7d0-69d5b44ce7de'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:49:17.477699 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-41d33f72-ac35-411b-b7d0-69d5b44ce7de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbbaee8a85c9455e9a31e0220990132f', 'request_id': 'req-41d33f72-ac35-411b-b7d0-69d5b44ce7de'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:49:17.501241 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-41d33f72-ac35-411b-b7d0-69d5b44ce7de admin admin] The token cache is being invalidate because project cbbaee8a85c9455e9a31e0220990132f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:49:17.501856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-41d33f72-ac35-411b-b7d0-69d5b44ce7de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:49:17.502027 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-41d33f72-ac35-411b-b7d0-69d5b44ce7de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:49:17.503365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 741/1510] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:49:17 2022] DELETE /identity/v3/projects/cbbaee8a85c9455e9a31e0220990132f => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:49:26.946377 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=62 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:49:27.499286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:50:57.673171 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78b0d8-4649-45c5-ab5e-910a44684b6a None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:50:57.673797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78b0d8-4649-45c5-ab5e-910a44684b6a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:50:57.673797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78b0d8-4649-45c5-ab5e-910a44684b6a None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:50:57.675701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 770/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 12:50:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:50:57.680857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32035c-7427-46b7-848a-b75485ca62a8 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:50:57.681303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32035c-7427-46b7-848a-b75485ca62a8 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:50:57.681408 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32035c-7427-46b7-848a-b75485ca62a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:50:57.760205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-9f32035c-7427-46b7-848a-b75485ca62a8 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:50:57.761421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-9f32035c-7427-46b7-848a-b75485ca62a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:50:57.863623 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 742/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 12:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:50:58.017430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78b0d8-4649-45c5-ab5e-910a44684b6a None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:50:58.020097 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ae78b0d8-4649-45c5-ab5e-910a44684b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:50:58.135958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78b0d8-4649-45c5-ab5e-910a44684b6a None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:50:58.141341 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd756b-5bff-471e-80f8-7eb9f78fc119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:50:58.141950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd756b-5bff-471e-80f8-7eb9f78fc119 service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:50:58.142231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd756b-5bff-471e-80f8-7eb9f78fc119 service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:50:58.142531 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd756b-5bff-471e-80f8-7eb9f78fc119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:50:58.150999 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd756b-5bff-471e-80f8-7eb9f78fc119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:50:58.152559 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abd756b-5bff-471e-80f8-7eb9f78fc119 service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:50:58.161768 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 771/1513] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:50:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:50:58.982532 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471eecf-7678-4455-a471-d1857d5eed28 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:50:58.983082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471eecf-7678-4455-a471-d1857d5eed28 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:50:58.983082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471eecf-7678-4455-a471-d1857d5eed28 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:50:58.984439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 743/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 12:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:50:58.989883 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d12e71-f7f5-4499-97d3-59365a7ad787 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:50:58.990319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d12e71-f7f5-4499-97d3-59365a7ad787 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:50:58.990428 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d12e71-f7f5-4499-97d3-59365a7ad787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:50:59.009570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-47d12e71-f7f5-4499-97d3-59365a7ad787 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:50:59.010802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-47d12e71-f7f5-4499-97d3-59365a7ad787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:50:59.018681 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 772/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 12:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:50:59.168205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471eecf-7678-4455-a471-d1857d5eed28 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:50:59.183226 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4471eecf-7678-4455-a471-d1857d5eed28 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:50:59.188741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5f7cd8-e396-4b35-9577-d10dd2b67b00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:50:59.189346 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f7cd8-e396-4b35-9577-d10dd2b67b00 service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:50:59.189651 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f7cd8-e396-4b35-9577-d10dd2b67b00 service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:50:59.189935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5f7cd8-e396-4b35-9577-d10dd2b67b00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:50:59.198655 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f7cd8-e396-4b35-9577-d10dd2b67b00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:50:59.200239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5f7cd8-e396-4b35-9577-d10dd2b67b00 service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:50:59.209695 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 744/1516] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:50:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:50:59.452847 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff41e8d-aa82-4332-96aa-13c0997b62e7 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:50:59.453055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff41e8d-aa82-4332-96aa-13c0997b62e7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:50:59.453324 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff41e8d-aa82-4332-96aa-13c0997b62e7 None None] PATH_INFO: `/v3` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:50:59.454678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 773/1517] 10.222.0.77 () {54 vars in 925 bytes} [Wed Aug 3 12:50:59 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:50:59.460552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d8740-c4d9-4fce-8bd0-fcd64fd9f230 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:50:59.460837 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d8740-c4d9-4fce-8bd0-fcd64fd9f230 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:50:59.461111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d8740-c4d9-4fce-8bd0-fcd64fd9f230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:50:59.519698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-637d8740-c4d9-4fce-8bd0-fcd64fd9f230 None None] MFA Rules not processed for user `5094263ac471481db48783b19192e01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:50:59.520906 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-637d8740-c4d9-4fce-8bd0-fcd64fd9f230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:50:59.537899 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:50:59.592377 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 745/1518] 10.222.0.77 () {58 vars in 1015 bytes} [Wed Aug 3 12:50:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:51:08.483232 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c8aef-b672-4d68-9d54-8cde9722446f None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:08.483743 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c8aef-b672-4d68-9d54-8cde9722446f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:08.483743 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c8aef-b672-4d68-9d54-8cde9722446f None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:08.485027 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 774/1519] 10.222.0.65 () {54 vars in 961 bytes} [Wed Aug 3 12:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:51:08.490210 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a7dfc-2141-4207-94b3-f07a193e778e None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:08.490669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a7dfc-2141-4207-94b3-f07a193e778e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:08.490800 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a7dfc-2141-4207-94b3-f07a193e778e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:08.510110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-4e5a7dfc-2141-4207-94b3-f07a193e778e None None] MFA Rules not processed for user `5094263ac471481db48783b19192e01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:51:08.511335 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-4e5a7dfc-2141-4207-94b3-f07a193e778e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:51:08.520466 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 746/1520] 10.222.0.65 () {58 vars in 1058 bytes} [Wed Aug 3 12:51:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:51:08.527592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c8aef-b672-4d68-9d54-8cde9722446f None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:51:08.540843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c8aef-b672-4d68-9d54-8cde9722446f None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:51:08.545737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849897f0-1a58-41bc-9455-215103a1d8ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5094263ac471481db48783b19192e01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:51:08.546348 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849897f0-1a58-41bc-9455-215103a1d8ad service placement] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:08.546660 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849897f0-1a58-41bc-9455-215103a1d8ad service placement] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:08.546924 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849897f0-1a58-41bc-9455-215103a1d8ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:08.555320 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849897f0-1a58-41bc-9455-215103a1d8ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:51:08.556866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849897f0-1a58-41bc-9455-215103a1d8ad service placement] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:51:08.567089 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 775/1521] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:51:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:51:10.431477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e937166-3349-4fe1-a181-93fa1f0cb57e None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:10.432062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e937166-3349-4fe1-a181-93fa1f0cb57e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:10.432062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e937166-3349-4fe1-a181-93fa1f0cb57e None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:10.433380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 747/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 12:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:51:10.438280 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0ccc6-f14b-4b41-8005-b5fcb8df0f4f None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:10.438739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0ccc6-f14b-4b41-8005-b5fcb8df0f4f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:10.438982 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0ccc6-f14b-4b41-8005-b5fcb8df0f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:10.458448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-8bf0ccc6-f14b-4b41-8005-b5fcb8df0f4f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:51:10.459666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8bf0ccc6-f14b-4b41-8005-b5fcb8df0f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:51:10.467442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 776/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 12:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:51:10.623313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e937166-3349-4fe1-a181-93fa1f0cb57e None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:51:10.638578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e937166-3349-4fe1-a181-93fa1f0cb57e None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:51:10.643921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1fdf3-db61-4378-a9ee-ba10e9e5bae5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:51:10.644523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1fdf3-db61-4378-a9ee-ba10e9e5bae5 service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:10.644797 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1fdf3-db61-4378-a9ee-ba10e9e5bae5 service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:10.645066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1fdf3-db61-4378-a9ee-ba10e9e5bae5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:10.653088 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1fdf3-db61-4378-a9ee-ba10e9e5bae5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:51:10.654678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1fdf3-db61-4378-a9ee-ba10e9e5bae5 service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:51:10.663965 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 748/1524] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:51:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:51:11.482036 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db44510-c28c-4371-8d98-903825df88ca None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:11.482601 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db44510-c28c-4371-8d98-903825df88ca None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:11.482701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db44510-c28c-4371-8d98-903825df88ca None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:11.483982 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 777/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 3 12:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:51:11.488811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e9f6a-b03c-4485-9176-318d54af8a6b None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:11.489246 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e9f6a-b03c-4485-9176-318d54af8a6b None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:11.489351 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e9f6a-b03c-4485-9176-318d54af8a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:11.508961 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-6e8e9f6a-b03c-4485-9176-318d54af8a6b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:51:11.510141 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6e8e9f6a-b03c-4485-9176-318d54af8a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:51:11.517841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 749/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 3 12:51:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:51:11.672244 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db44510-c28c-4371-8d98-903825df88ca None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:51:11.687344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db44510-c28c-4371-8d98-903825df88ca None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:51:11.692856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cf88f-a723-4b6a-82b3-220e86ff43e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:51:11.693459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cf88f-a723-4b6a-82b3-220e86ff43e3 service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:11.693742 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cf88f-a723-4b6a-82b3-220e86ff43e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:11.694018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cf88f-a723-4b6a-82b3-220e86ff43e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:11.703120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cf88f-a723-4b6a-82b3-220e86ff43e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:51:11.704973 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962cf88f-a723-4b6a-82b3-220e86ff43e3 service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:51:11.714869 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 778/1527] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:51:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:51:15.478212 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cc4ae-9ccd-4301-ad42-06a6bfa96900 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:15.478828 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cc4ae-9ccd-4301-ad42-06a6bfa96900 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:15.478828 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8cc4ae-9ccd-4301-ad42-06a6bfa96900 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:15.480212 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 750/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 12:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:51:15.485400 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393247b-3e0b-4743-b54c-58c5c0fa855f None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:15.485858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393247b-3e0b-4743-b54c-58c5c0fa855f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:15.485965 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c393247b-3e0b-4743-b54c-58c5c0fa855f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:15.505859 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-c393247b-3e0b-4743-b54c-58c5c0fa855f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:51:15.507083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c393247b-3e0b-4743-b54c-58c5c0fa855f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:51:15.514942 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 779/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 12:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:51:15.587611 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cc4ae-9ccd-4301-ad42-06a6bfa96900 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:51:15.589327 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ef8cc4ae-9ccd-4301-ad42-06a6bfa96900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:51:15.667819 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8cc4ae-9ccd-4301-ad42-06a6bfa96900 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:51:15.671415 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f33358-ef90-48c5-aa7f-4dc71973e649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:51:15.672021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f33358-ef90-48c5-aa7f-4dc71973e649 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:15.672299 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f33358-ef90-48c5-aa7f-4dc71973e649 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:15.672570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f33358-ef90-48c5-aa7f-4dc71973e649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:15.681670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f33358-ef90-48c5-aa7f-4dc71973e649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:51:15.683122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f33358-ef90-48c5-aa7f-4dc71973e649 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:51:15.692414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 751/1530] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:51:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:51:17.666941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d029ab-2078-4c4e-9be7-a42fb6ef29a5 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:17.667512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d029ab-2078-4c4e-9be7-a42fb6ef29a5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:17.667512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d029ab-2078-4c4e-9be7-a42fb6ef29a5 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:17.668808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 780/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 12:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:51:17.673886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65f185-85dd-493b-a397-f54a4e536cd5 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:17.674323 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65f185-85dd-493b-a397-f54a4e536cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:17.674431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65f185-85dd-493b-a397-f54a4e536cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:17.694294 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-ac65f185-85dd-493b-a397-f54a4e536cd5 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:51:17.695520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ac65f185-85dd-493b-a397-f54a4e536cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:51:17.703242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 752/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 12:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:51:17.738925 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d029ab-2078-4c4e-9be7-a42fb6ef29a5 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:51:17.740607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b0d029ab-2078-4c4e-9be7-a42fb6ef29a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:51:17.753610 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d029ab-2078-4c4e-9be7-a42fb6ef29a5 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:51:17.757731 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e00984-13ed-41d4-bcf6-7a9c58e1c396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:51:17.758336 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e00984-13ed-41d4-bcf6-7a9c58e1c396 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:17.758653 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e00984-13ed-41d4-bcf6-7a9c58e1c396 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:17.758930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e00984-13ed-41d4-bcf6-7a9c58e1c396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:17.767773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e00984-13ed-41d4-bcf6-7a9c58e1c396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:51:17.769184 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e00984-13ed-41d4-bcf6-7a9c58e1c396 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:51:17.779182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 781/1533] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:51:21.224620 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1d77c-874e-447c-b915-6fe9c725792a None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:21.225072 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1d77c-874e-447c-b915-6fe9c725792a None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:21.225185 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1d77c-874e-447c-b915-6fe9c725792a None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:21.226548 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 753/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 12:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:51:21.231661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c91190-3637-463e-a22a-911b1061a971 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:21.232106 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c91190-3637-463e-a22a-911b1061a971 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:21.232217 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c91190-3637-463e-a22a-911b1061a971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:21.252419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-95c91190-3637-463e-a22a-911b1061a971 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:51:21.253619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-95c91190-3637-463e-a22a-911b1061a971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:51:21.261426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 782/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 12:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:51:21.563401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1d77c-874e-447c-b915-6fe9c725792a None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:51:21.565089 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-3af1d77c-874e-447c-b915-6fe9c725792a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:51:21.629545 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:51:21.642744 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1d77c-874e-447c-b915-6fe9c725792a None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:51:21.646863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b22c1-072f-434b-abe6-bf1de9ce226a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:51:21.647480 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b22c1-072f-434b-abe6-bf1de9ce226a service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:21.647756 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b22c1-072f-434b-abe6-bf1de9ce226a service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:21.648029 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b22c1-072f-434b-abe6-bf1de9ce226a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:21.656872 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b22c1-072f-434b-abe6-bf1de9ce226a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:51:21.658278 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b22c1-072f-434b-abe6-bf1de9ce226a service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:51:21.668112 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 754/1536] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:51:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:51:23.665398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d4f1a-881e-433f-a93e-7aa66654f219 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:23.665977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d4f1a-881e-433f-a93e-7aa66654f219 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:23.665977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d4f1a-881e-433f-a93e-7aa66654f219 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:23.667365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 783/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Aug 3 12:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:51:23.672760 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc768ba-5107-4f9c-aafe-c48fba9eff54 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:23.673200 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc768ba-5107-4f9c-aafe-c48fba9eff54 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:23.673297 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc768ba-5107-4f9c-aafe-c48fba9eff54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:23.693413 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-abc768ba-5107-4f9c-aafe-c48fba9eff54 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:51:23.694660 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-abc768ba-5107-4f9c-aafe-c48fba9eff54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:51:23.702447 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 755/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Aug 3 12:51:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:51:24.002108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d4f1a-881e-433f-a93e-7aa66654f219 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:51:24.003776 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d68d4f1a-881e-433f-a93e-7aa66654f219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:51:24.016742 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d4f1a-881e-433f-a93e-7aa66654f219 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:51:24.020741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c68748-b648-4d04-a606-c9afa35ea94c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:51:24.021340 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c68748-b648-4d04-a606-c9afa35ea94c service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:51:24.021613 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c68748-b648-4d04-a606-c9afa35ea94c service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:51:24.021883 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c68748-b648-4d04-a606-c9afa35ea94c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:51:24.030637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c68748-b648-4d04-a606-c9afa35ea94c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:51:24.032041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c68748-b648-4d04-a606-c9afa35ea94c service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:51:24.041900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 784/1539] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:51:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:51:33.688701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=11 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:51:34.037851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:50.760974 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14b9a4-241f-411c-96f2-574f5413b562 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:50.761637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14b9a4-241f-411c-96f2-574f5413b562 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:50.761637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14b9a4-241f-411c-96f2-574f5413b562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:50.785611 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-7d14b9a4-241f-411c-96f2-574f5413b562 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:50.786947 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-7d14b9a4-241f-411c-96f2-574f5413b562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:50.798586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 756/1540] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:50.807511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc15ba-55de-4f36-8cf4-f6eb9e13ee20 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:50.808458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc15ba-55de-4f36-8cf4-f6eb9e13ee20 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:50.808458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc15ba-55de-4f36-8cf4-f6eb9e13ee20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:50.829355 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2ddd7-9607-4473-9825-2a1956aa775d None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:50.829706 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2ddd7-9607-4473-9825-2a1956aa775d None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:50.829918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2ddd7-9607-4473-9825-2a1956aa775d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:50.841130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-81bc15ba-55de-4f36-8cf4-f6eb9e13ee20 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:50.842355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-81bc15ba-55de-4f36-8cf4-f6eb9e13ee20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:50.857342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 785/1541] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:50.858281 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a8b2ddd7-9607-4473-9825-2a1956aa775d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:50.858644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9e3da-85f8-44b7-b41a-093cbb8eb76a None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:50.858933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9e3da-85f8-44b7-b41a-093cbb8eb76a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:50.859206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a9e3da-85f8-44b7-b41a-093cbb8eb76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:50.859545 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a8b2ddd7-9607-4473-9825-2a1956aa775d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:50.873973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 757/1542] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:52:50.875887 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2ddd7-9607-4473-9825-2a1956aa775d None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:50.894169 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2ddd7-9607-4473-9825-2a1956aa775d None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:50.898357 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-53a9e3da-85f8-44b7-b41a-093cbb8eb76a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:50.899600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-53a9e3da-85f8-44b7-b41a-093cbb8eb76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:50.904686 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef420c4-0533-4c09-8c8b-b0db6d8e2cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:50.905319 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef420c4-0533-4c09-8c8b-b0db6d8e2cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:50.905598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef420c4-0533-4c09-8c8b-b0db6d8e2cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:50.905866 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef420c4-0533-4c09-8c8b-b0db6d8e2cf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:50.906756 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef420c4-0533-4c09-8c8b-b0db6d8e2cf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:52:50.907190 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef420c4-0533-4c09-8c8b-b0db6d8e2cf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:50.908798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef420c4-0533-4c09-8c8b-b0db6d8e2cf6 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:50.910542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 786/1543] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:50.911775 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa38ccf-932a-4642-b642-8837f6c1e269 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:50.912058 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa38ccf-932a-4642-b642-8837f6c1e269 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:50.912327 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa38ccf-932a-4642-b642-8837f6c1e269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:50.929406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 758/1544] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:52:50.930838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50abee-d08e-4e0f-9af7-3600f29827e9 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:50.931132 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50abee-d08e-4e0f-9af7-3600f29827e9 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:50.931411 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50abee-d08e-4e0f-9af7-3600f29827e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:50.937070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-daa38ccf-932a-4642-b642-8837f6c1e269 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:50.938511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-daa38ccf-932a-4642-b642-8837f6c1e269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:50.947911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 787/1545] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:50.949142 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d510edb-29e3-499f-9281-e1331ed8c71b None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:50.949424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d510edb-29e3-499f-9281-e1331ed8c71b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:50.949696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d510edb-29e3-499f-9281-e1331ed8c71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:50.955603 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-6f50abee-d08e-4e0f-9af7-3600f29827e9 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:50.957165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6f50abee-d08e-4e0f-9af7-3600f29827e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:50.967169 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 759/1546] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:50.968611 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d735039-c089-4998-9bc6-2d6135f827a5 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:50.969000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d735039-c089-4998-9bc6-2d6135f827a5 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:50.969490 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d735039-c089-4998-9bc6-2d6135f827a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:50.976301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-8d510edb-29e3-499f-9281-e1331ed8c71b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:50.978358 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8d510edb-29e3-499f-9281-e1331ed8c71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:50.996925 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 788/1547] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:50.997636 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d510edb-29e3-499f-9281-e1331ed8c71b None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.008702 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-6d735039-c089-4998-9bc6-2d6135f827a5 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:51.009799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d510edb-29e3-499f-9281-e1331ed8c71b None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.011358 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6d735039-c089-4998-9bc6-2d6135f827a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:51.020212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07da381-4881-4b86-ae6c-4e4b7debd457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.020842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07da381-4881-4b86-ae6c-4e4b7debd457 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.021122 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07da381-4881-4b86-ae6c-4e4b7debd457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.021394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07da381-4881-4b86-ae6c-4e4b7debd457 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.022069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07da381-4881-4b86-ae6c-4e4b7debd457 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.023841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 760/1548] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:51.024736 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07da381-4881-4b86-ae6c-4e4b7debd457 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.024861 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d735039-c089-4998-9bc6-2d6135f827a5 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.038733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d735039-c089-4998-9bc6-2d6135f827a5 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.044463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb25f8-94c9-4738-91f6-4f097585c7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.045085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb25f8-94c9-4738-91f6-4f097585c7a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.045366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb25f8-94c9-4738-91f6-4f097585c7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.045639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb25f8-94c9-4738-91f6-4f097585c7a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.046340 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb25f8-94c9-4738-91f6-4f097585c7a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:52:51.046739 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb25f8-94c9-4738-91f6-4f097585c7a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.048491 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb25f8-94c9-4738-91f6-4f097585c7a7 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.068832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 761/1549] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:52:51.068832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d18299-5abd-491c-9164-a3dd825ec7fc None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.068832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d18299-5abd-491c-9164-a3dd825ec7fc None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.068832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d18299-5abd-491c-9164-a3dd825ec7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.075968 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 789/1550] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:52:50 2022] POST /identity/v3/projects => generated 365 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:51.077317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff51ff-78d3-46c5-a2c9-3e7c38422ab8 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.077667 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff51ff-78d3-46c5-a2c9-3e7c38422ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.078196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbff51ff-78d3-46c5-a2c9-3e7c38422ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.083974 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-66d18299-5abd-491c-9164-a3dd825ec7fc None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:51.085182 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-66d18299-5abd-491c-9164-a3dd825ec7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:51.098108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-cbff51ff-78d3-46c5-a2c9-3e7c38422ab8 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:51.099302 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-cbff51ff-78d3-46c5-a2c9-3e7c38422ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:51.143751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 762/1551] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:51.145528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f17e4f-b988-4bd7-8d45-73131655e50b None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.145815 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f17e4f-b988-4bd7-8d45-73131655e50b None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.146088 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f17e4f-b988-4bd7-8d45-73131655e50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.162239 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 790/1552] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:51.166007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbe421-699b-4fce-aa9e-74476c459a91 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.166007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbe421-699b-4fce-aa9e-74476c459a91 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.166007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfbe421-699b-4fce-aa9e-74476c459a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.166274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-11f17e4f-b988-4bd7-8d45-73131655e50b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:51.166850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-11f17e4f-b988-4bd7-8d45-73131655e50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:51.183603 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 763/1553] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:51.183603 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f17e4f-b988-4bd7-8d45-73131655e50b None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.187330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f17e4f-b988-4bd7-8d45-73131655e50b None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.193613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2cf53-f6f7-478c-9f8d-6fbddc78bf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.194450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2cf53-f6f7-478c-9f8d-6fbddc78bf78 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.194781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2cf53-f6f7-478c-9f8d-6fbddc78bf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.195068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-6dfbe421-699b-4fce-aa9e-74476c459a91 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:51.195331 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2cf53-f6f7-478c-9f8d-6fbddc78bf78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.195827 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2cf53-f6f7-478c-9f8d-6fbddc78bf78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:52:51.196094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2cf53-f6f7-478c-9f8d-6fbddc78bf78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.196582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-6dfbe421-699b-4fce-aa9e-74476c459a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:51.197952 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd2cf53-f6f7-478c-9f8d-6fbddc78bf78 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.205571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 791/1554] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:51.207363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bf4d5-082d-40fa-93a4-ff211e5cb94a None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.207762 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bf4d5-082d-40fa-93a4-ff211e5cb94a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.208038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bf4d5-082d-40fa-93a4-ff211e5cb94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.217050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 764/1555] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:52:51.217960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2cf53-f6f7-478c-9f8d-6fbddc78bf78 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.227663 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-f31bf4d5-082d-40fa-93a4-ff211e5cb94a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:51.228865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f31bf4d5-082d-40fa-93a4-ff211e5cb94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:51.229173 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2cf53-f6f7-478c-9f8d-6fbddc78bf78 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.233938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a1ac1-7d05-43ca-b53f-b0a6bdf5711b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.234607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a1ac1-7d05-43ca-b53f-b0a6bdf5711b admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.234913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a1ac1-7d05-43ca-b53f-b0a6bdf5711b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.235184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a1ac1-7d05-43ca-b53f-b0a6bdf5711b admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.235838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a1ac1-7d05-43ca-b53f-b0a6bdf5711b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.236825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 792/1556] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:51.236949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: WARNING py.warnings [None req-0e4a1ac1-7d05-43ca-b53f-b0a6bdf5711b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 12:52:51.236949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: warnings.warn(msg) Aug 03 12:52:51.236949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: Aug 03 12:52:51.237559 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bf4d5-082d-40fa-93a4-ff211e5cb94a None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.238034 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a1ac1-7d05-43ca-b53f-b0a6bdf5711b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.247344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bf4d5-082d-40fa-93a4-ff211e5cb94a None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.252020 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0f3d7-016c-4040-9599-cf1cab545bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.252625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0f3d7-016c-4040-9599-cf1cab545bef admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.252908 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0f3d7-016c-4040-9599-cf1cab545bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.253179 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0f3d7-016c-4040-9599-cf1cab545bef admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.254381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0f3d7-016c-4040-9599-cf1cab545bef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.255438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: WARNING py.warnings [None req-6aa0f3d7-016c-4040-9599-cf1cab545bef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 12:52:51.255438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: warnings.warn(msg) Aug 03 12:52:51.255438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: Aug 03 12:52:51.256621 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0f3d7-016c-4040-9599-cf1cab545bef admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.270280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 765/1557] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:51.270280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a1ac1-7d05-43ca-b53f-b0a6bdf5711b admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.285442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 793/1558] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/users => generated 283 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:51.285442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0f3d7-016c-4040-9599-cf1cab545bef admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.292888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0f3d7-016c-4040-9599-cf1cab545bef admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.331432 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a1ac1-7d05-43ca-b53f-b0a6bdf5711b admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.336859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97764e62-babb-42cf-8c19-850f77c95078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.337478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97764e62-babb-42cf-8c19-850f77c95078 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.337759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97764e62-babb-42cf-8c19-850f77c95078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.338035 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97764e62-babb-42cf-8c19-850f77c95078 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.338761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97764e62-babb-42cf-8c19-850f77c95078 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:52:51.339066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97764e62-babb-42cf-8c19-850f77c95078 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.340659 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97764e62-babb-42cf-8c19-850f77c95078 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.349283 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:51.352307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd923f9d-f025-429d-b351-4462f25bfdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.353013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd923f9d-f025-429d-b351-4462f25bfdb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.353372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd923f9d-f025-429d-b351-4462f25bfdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.353772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd923f9d-f025-429d-b351-4462f25bfdb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.354518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd923f9d-f025-429d-b351-4462f25bfdb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:52:51.354890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 766/1559] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:52:51.355014 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd923f9d-f025-429d-b351-4462f25bfdb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.355976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: WARNING py.warnings [None req-dd923f9d-f025-429d-b351-4462f25bfdb9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 03 12:52:51.355976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: warnings.warn(msg) Aug 03 12:52:51.355976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: Aug 03 12:52:51.356601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d80f11-7fb2-44c8-87e6-e5e6c33e3529 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.357057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d80f11-7fb2-44c8-87e6-e5e6c33e3529 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.357321 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd923f9d-f025-429d-b351-4462f25bfdb9 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.357570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d80f11-7fb2-44c8-87e6-e5e6c33e3529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.373110 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 794/1560] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:52:51.380369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef4f05-b929-47f4-93cf-2151672a7308 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.380855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef4f05-b929-47f4-93cf-2151672a7308 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.381310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef4f05-b929-47f4-93cf-2151672a7308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.387386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-91d80f11-7fb2-44c8-87e6-e5e6c33e3529 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:51.388745 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-91d80f11-7fb2-44c8-87e6-e5e6c33e3529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:51.396614 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:51.397559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 767/1561] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:51.398432 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d80f11-7fb2-44c8-87e6-e5e6c33e3529 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.407673 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-66ef4f05-b929-47f4-93cf-2151672a7308 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:51.408923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d80f11-7fb2-44c8-87e6-e5e6c33e3529 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.409052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-66ef4f05-b929-47f4-93cf-2151672a7308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:51.413677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d017b-157c-4cd6-ad56-afac526a6163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.414276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d017b-157c-4cd6-ad56-afac526a6163 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.414582 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d017b-157c-4cd6-ad56-afac526a6163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.414858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d017b-157c-4cd6-ad56-afac526a6163 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.415515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d017b-157c-4cd6-ad56-afac526a6163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.417091 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 795/1562] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:51.417507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d017b-157c-4cd6-ad56-afac526a6163 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.418338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99ae81-a75e-465d-95bc-e436d4f3a23e None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.418667 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99ae81-a75e-465d-95bc-e436d4f3a23e None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.418944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99ae81-a75e-465d-95bc-e436d4f3a23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.439885 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1b99ae81-a75e-465d-95bc-e436d4f3a23e None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:51.441604 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1b99ae81-a75e-465d-95bc-e436d4f3a23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:51.446258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 768/1563] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:51.447310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d017b-157c-4cd6-ad56-afac526a6163 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.449884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 796/1564] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:51.450851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99ae81-a75e-465d-95bc-e436d4f3a23e None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.460730 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99ae81-a75e-465d-95bc-e436d4f3a23e None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.512909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d017b-157c-4cd6-ad56-afac526a6163 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.516643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbc1eb-9b0e-4b3f-883e-0271cea1c044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.517260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbc1eb-9b0e-4b3f-883e-0271cea1c044 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.517535 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbc1eb-9b0e-4b3f-883e-0271cea1c044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.517638 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c9988-54b2-4c85-bbac-78b914c7ad40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.517812 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbc1eb-9b0e-4b3f-883e-0271cea1c044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.518179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c9988-54b2-4c85-bbac-78b914c7ad40 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.518499 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c9988-54b2-4c85-bbac-78b914c7ad40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.518620 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbc1eb-9b0e-4b3f-883e-0271cea1c044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.518792 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c9988-54b2-4c85-bbac-78b914c7ad40 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.520039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c9988-54b2-4c85-bbac-78b914c7ad40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.520738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefbc1eb-9b0e-4b3f-883e-0271cea1c044 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.521931 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c9988-54b2-4c85-bbac-78b914c7ad40 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.530376 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 797/1565] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:51.531266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbc1eb-9b0e-4b3f-883e-0271cea1c044 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.541251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbc1eb-9b0e-4b3f-883e-0271cea1c044 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.547525 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 769/1566] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/users => generated 302 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:51.547525 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c9988-54b2-4c85-bbac-78b914c7ad40 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.549393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bd166-5a1e-46b7-9878-39ae30a0ea3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.550016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bd166-5a1e-46b7-9878-39ae30a0ea3b admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.550273 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bd166-5a1e-46b7-9878-39ae30a0ea3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.550573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bd166-5a1e-46b7-9878-39ae30a0ea3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.551262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17bd166-5a1e-46b7-9878-39ae30a0ea3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.553387 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17bd166-5a1e-46b7-9878-39ae30a0ea3b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.555125 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c9988-54b2-4c85-bbac-78b914c7ad40 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.560440 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb7c9d-647a-4fa4-80ca-f977e97689f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.561045 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb7c9d-647a-4fa4-80ca-f977e97689f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.561328 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb7c9d-647a-4fa4-80ca-f977e97689f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.561604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb7c9d-647a-4fa4-80ca-f977e97689f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.562272 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb7c9d-647a-4fa4-80ca-f977e97689f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.564217 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb7c9d-647a-4fa4-80ca-f977e97689f8 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.583991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 798/1567] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:51.584915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bd166-5a1e-46b7-9878-39ae30a0ea3b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.597991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 770/1568] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:51.598864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb7c9d-647a-4fa4-80ca-f977e97689f8 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.655103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bd166-5a1e-46b7-9878-39ae30a0ea3b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.659737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4404d2-05ac-4ef0-92f0-4d3ded0696e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.660854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4404d2-05ac-4ef0-92f0-4d3ded0696e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.661130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4404d2-05ac-4ef0-92f0-4d3ded0696e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.661400 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4404d2-05ac-4ef0-92f0-4d3ded0696e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.662092 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4404d2-05ac-4ef0-92f0-4d3ded0696e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:52:51.662389 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4404d2-05ac-4ef0-92f0-4d3ded0696e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.664006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb7c9d-647a-4fa4-80ca-f977e97689f8 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.664137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4404d2-05ac-4ef0-92f0-4d3ded0696e1 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.668829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7df4cf-6e78-4885-a0b5-dd555e60c28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.669473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7df4cf-6e78-4885-a0b5-dd555e60c28b admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.669754 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7df4cf-6e78-4885-a0b5-dd555e60c28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.670029 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7df4cf-6e78-4885-a0b5-dd555e60c28b admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.670772 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7df4cf-6e78-4885-a0b5-dd555e60c28b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:52:51.671079 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7df4cf-6e78-4885-a0b5-dd555e60c28b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.672657 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7df4cf-6e78-4885-a0b5-dd555e60c28b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.675215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 799/1569] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:52:51.676000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4404d2-05ac-4ef0-92f0-4d3ded0696e1 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.688076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4404d2-05ac-4ef0-92f0-4d3ded0696e1 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.688254 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 771/1570] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:52:51.689124 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7df4cf-6e78-4885-a0b5-dd555e60c28b admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.692738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b49231-9667-41ad-a6d4-dddbec64e647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.693338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b49231-9667-41ad-a6d4-dddbec64e647 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.693619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b49231-9667-41ad-a6d4-dddbec64e647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.693894 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b49231-9667-41ad-a6d4-dddbec64e647 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.694580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b49231-9667-41ad-a6d4-dddbec64e647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.696646 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b49231-9667-41ad-a6d4-dddbec64e647 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.700077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7df4cf-6e78-4885-a0b5-dd555e60c28b admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.704922 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c45d38-c63b-4ce6-86d2-384382b1a826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.705525 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c45d38-c63b-4ce6-86d2-384382b1a826 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.705810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c45d38-c63b-4ce6-86d2-384382b1a826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.706082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c45d38-c63b-4ce6-86d2-384382b1a826 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.707332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c45d38-c63b-4ce6-86d2-384382b1a826 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:52:51.707636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c45d38-c63b-4ce6-86d2-384382b1a826 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.709222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c45d38-c63b-4ce6-86d2-384382b1a826 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.719285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 800/1571] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:51.720272 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b49231-9667-41ad-a6d4-dddbec64e647 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.723455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 772/1572] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:52:51.724266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c45d38-c63b-4ce6-86d2-384382b1a826 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.734569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c45d38-c63b-4ce6-86d2-384382b1a826 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.735750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b49231-9667-41ad-a6d4-dddbec64e647 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.739375 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b4024-0746-4d42-a164-153dce85e4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.739973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b4024-0746-4d42-a164-153dce85e4db admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.740256 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b4024-0746-4d42-a164-153dce85e4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.740364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4fb60-9e9d-4cd4-93a4-33012b9e0b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.740535 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b4024-0746-4d42-a164-153dce85e4db admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.740855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4fb60-9e9d-4cd4-93a4-33012b9e0b69 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.741132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4fb60-9e9d-4cd4-93a4-33012b9e0b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.741242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b4024-0746-4d42-a164-153dce85e4db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.741403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4fb60-9e9d-4cd4-93a4-33012b9e0b69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.742088 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4fb60-9e9d-4cd4-93a4-33012b9e0b69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.742862 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b4024-0746-4d42-a164-153dce85e4db admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.743729 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f4fb60-9e9d-4cd4-93a4-33012b9e0b69 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.753238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 773/1573] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:51.754153 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b4024-0746-4d42-a164-153dce85e4db admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.754738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 801/1574] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:51.755429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4fb60-9e9d-4cd4-93a4-33012b9e0b69 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.764467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b4024-0746-4d42-a164-153dce85e4db admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.767603 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f4fb60-9e9d-4cd4-93a4-33012b9e0b69 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.769694 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74405ffb-05a1-4861-a0e9-de0d09abd3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.770302 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74405ffb-05a1-4861-a0e9-de0d09abd3e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.770613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74405ffb-05a1-4861-a0e9-de0d09abd3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.770935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74405ffb-05a1-4861-a0e9-de0d09abd3e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.771547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74405ffb-05a1-4861-a0e9-de0d09abd3e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.773045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55570218-4ab4-463d-b9a7-f6fb72eff25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.773454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74405ffb-05a1-4861-a0e9-de0d09abd3e7 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.773790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55570218-4ab4-463d-b9a7-f6fb72eff25f admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.774143 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55570218-4ab4-463d-b9a7-f6fb72eff25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.774505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55570218-4ab4-463d-b9a7-f6fb72eff25f admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.775261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55570218-4ab4-463d-b9a7-f6fb72eff25f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.777210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55570218-4ab4-463d-b9a7-f6fb72eff25f admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.782366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:51.795839 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 774/1575] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:51.796605 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74405ffb-05a1-4861-a0e9-de0d09abd3e7 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.803605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 802/1576] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/users => generated 295 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:51.803605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55570218-4ab4-463d-b9a7-f6fb72eff25f admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.808750 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74405ffb-05a1-4861-a0e9-de0d09abd3e7 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.810823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55570218-4ab4-463d-b9a7-f6fb72eff25f admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.813506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c663bb-dd4e-4744-a556-9c1a40f6a1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.814103 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c663bb-dd4e-4744-a556-9c1a40f6a1df admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.814389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c663bb-dd4e-4744-a556-9c1a40f6a1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.814695 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c663bb-dd4e-4744-a556-9c1a40f6a1df admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.815350 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c663bb-dd4e-4744-a556-9c1a40f6a1df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.815446 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287e3cf-f68f-4d34-b257-972db590384d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.816062 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287e3cf-f68f-4d34-b257-972db590384d admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.816343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287e3cf-f68f-4d34-b257-972db590384d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.816618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287e3cf-f68f-4d34-b257-972db590384d admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.817297 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c663bb-dd4e-4744-a556-9c1a40f6a1df admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.817396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287e3cf-f68f-4d34-b257-972db590384d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.819563 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287e3cf-f68f-4d34-b257-972db590384d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.836216 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:51.844821 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 775/1577] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:51.845542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c663bb-dd4e-4744-a556-9c1a40f6a1df admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.853080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 803/1578] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:51.854169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287e3cf-f68f-4d34-b257-972db590384d admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.909578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c663bb-dd4e-4744-a556-9c1a40f6a1df admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.920508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287e3cf-f68f-4d34-b257-972db590384d admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:51.925237 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c45263-cb2b-4df9-a8d6-8df3418a4072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.925877 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45263-cb2b-4df9-a8d6-8df3418a4072 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.926157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45263-cb2b-4df9-a8d6-8df3418a4072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.926446 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45263-cb2b-4df9-a8d6-8df3418a4072 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.927377 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c45263-cb2b-4df9-a8d6-8df3418a4072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.929269 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c45263-cb2b-4df9-a8d6-8df3418a4072 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.957504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 804/1579] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:52:51 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:51.958304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c45263-cb2b-4df9-a8d6-8df3418a4072 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.967574 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bda106-360f-4a96-a012-a25171001ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:51.968190 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bda106-360f-4a96-a012-a25171001ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:51.968473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bda106-360f-4a96-a012-a25171001ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:51.968743 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bda106-360f-4a96-a012-a25171001ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:51.969437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bda106-360f-4a96-a012-a25171001ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:51.971070 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bda106-360f-4a96-a012-a25171001ef3 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:51.980828 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 776/1580] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:51.981447 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bda106-360f-4a96-a012-a25171001ef3 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:51.990610 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bda106-360f-4a96-a012-a25171001ef3 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.020241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c45263-cb2b-4df9-a8d6-8df3418a4072 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.025328 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eaa0a4-7b3f-42cd-9e3b-86a03e019357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.025950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaa0a4-7b3f-42cd-9e3b-86a03e019357 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.026235 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaa0a4-7b3f-42cd-9e3b-86a03e019357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.026540 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eaa0a4-7b3f-42cd-9e3b-86a03e019357 admin admin] PATH_INFO: `/v3/projects/6ca733bcc6dd48afb6e0fcb62b56b9f9/users/4a4d8a2066094a56afbae522f0629473/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.040972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eaa0a4-7b3f-42cd-9e3b-86a03e019357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ca733bcc6dd48afb6e0fcb62b56b9f9, user_id=4a4d8a2066094a56afbae522f0629473, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.043653 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eaa0a4-7b3f-42cd-9e3b-86a03e019357 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.045258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72b03d-21a9-4f67-8e04-f0516b3bbe59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.045875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72b03d-21a9-4f67-8e04-f0516b3bbe59 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.046152 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72b03d-21a9-4f67-8e04-f0516b3bbe59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.046422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72b03d-21a9-4f67-8e04-f0516b3bbe59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.047149 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72b03d-21a9-4f67-8e04-f0516b3bbe59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.049154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72b03d-21a9-4f67-8e04-f0516b3bbe59 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.057589 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 805/1581] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:52:51 2022] PUT /identity/v3/projects/6ca733bcc6dd48afb6e0fcb62b56b9f9/users/4a4d8a2066094a56afbae522f0629473/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:52.058773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eaa0a4-7b3f-42cd-9e3b-86a03e019357 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.062762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 777/1582] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:51 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:52.062762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72b03d-21a9-4f67-8e04-f0516b3bbe59 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.073357 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72b03d-21a9-4f67-8e04-f0516b3bbe59 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.079637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:52.134517 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eaa0a4-7b3f-42cd-9e3b-86a03e019357 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.137579 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db917c75-2d10-4893-a40f-c87bbed663ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.138346 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db917c75-2d10-4893-a40f-c87bbed663ea admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.138725 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db917c75-2d10-4893-a40f-c87bbed663ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.138997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db917c75-2d10-4893-a40f-c87bbed663ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.139129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a474594-4df5-480a-9027-416656bb6381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.139696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db917c75-2d10-4893-a40f-c87bbed663ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.139827 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a474594-4df5-480a-9027-416656bb6381 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.140017 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a474594-4df5-480a-9027-416656bb6381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.140290 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a474594-4df5-480a-9027-416656bb6381 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.140989 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a474594-4df5-480a-9027-416656bb6381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.141296 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db917c75-2d10-4893-a40f-c87bbed663ea admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.142660 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a474594-4df5-480a-9027-416656bb6381 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.144870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:52.151988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 778/1583] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:52.153032 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db917c75-2d10-4893-a40f-c87bbed663ea admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.157320 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 806/1584] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:52.158152 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a474594-4df5-480a-9027-416656bb6381 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.163476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db917c75-2d10-4893-a40f-c87bbed663ea admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.168382 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a474594-4df5-480a-9027-416656bb6381 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.168651 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62295d3d-8b66-40d0-b4af-f43604a83d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.169280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62295d3d-8b66-40d0-b4af-f43604a83d2b admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.169615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62295d3d-8b66-40d0-b4af-f43604a83d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.169896 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62295d3d-8b66-40d0-b4af-f43604a83d2b admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.170588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62295d3d-8b66-40d0-b4af-f43604a83d2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.172512 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62295d3d-8b66-40d0-b4af-f43604a83d2b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.173000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ecf982-eca6-46f4-86dc-93cc1bee984c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.173619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecf982-eca6-46f4-86dc-93cc1bee984c admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.173897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecf982-eca6-46f4-86dc-93cc1bee984c admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.174166 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecf982-eca6-46f4-86dc-93cc1bee984c admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.174869 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ecf982-eca6-46f4-86dc-93cc1bee984c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.177688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ecf982-eca6-46f4-86dc-93cc1bee984c admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.195396 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 779/1585] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:52:52 2022] POST /identity/v3/users => generated 288 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:52.198563 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 807/1586] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:52:52 2022] POST /identity/v3/users => generated 295 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:52.199819 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ecf982-eca6-46f4-86dc-93cc1bee984c admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.200566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62295d3d-8b66-40d0-b4af-f43604a83d2b admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.210137 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62295d3d-8b66-40d0-b4af-f43604a83d2b admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.210274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ecf982-eca6-46f4-86dc-93cc1bee984c admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.214752 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea084e4-c7dc-4bf0-ad9e-2c80983657ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.214901 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87363fcf-ab91-419b-aacd-5c9bc5b59296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.215346 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea084e4-c7dc-4bf0-ad9e-2c80983657ef admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.215451 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87363fcf-ab91-419b-aacd-5c9bc5b59296 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.215626 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea084e4-c7dc-4bf0-ad9e-2c80983657ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.215744 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87363fcf-ab91-419b-aacd-5c9bc5b59296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.215903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea084e4-c7dc-4bf0-ad9e-2c80983657ef admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.216007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87363fcf-ab91-419b-aacd-5c9bc5b59296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.216556 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea084e4-c7dc-4bf0-ad9e-2c80983657ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.216662 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87363fcf-ab91-419b-aacd-5c9bc5b59296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.218245 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87363fcf-ab91-419b-aacd-5c9bc5b59296 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.218542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea084e4-c7dc-4bf0-ad9e-2c80983657ef admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.233041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 808/1587] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:52.233041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87363fcf-ab91-419b-aacd-5c9bc5b59296 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.241616 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87363fcf-ab91-419b-aacd-5c9bc5b59296 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.246513 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 780/1588] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:52:52 2022] POST /identity/v3/users => generated 282 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:52.247258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca94a3-59a8-4f48-a543-ee05fd034c73 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.247544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca94a3-59a8-4f48-a543-ee05fd034c73 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.247812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca94a3-59a8-4f48-a543-ee05fd034c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.249387 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d02573-24d3-4d51-a73b-fa802415ad62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.250049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d02573-24d3-4d51-a73b-fa802415ad62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.250386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d02573-24d3-4d51-a73b-fa802415ad62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.250645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d02573-24d3-4d51-a73b-fa802415ad62 admin admin] PATH_INFO: `/v3/projects/dbcdc99798934d2e94b52d1806d49841/users/d32cb4e506d5426cb05fb9c1b5702ab0/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.265519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d02573-24d3-4d51-a73b-fa802415ad62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbcdc99798934d2e94b52d1806d49841, user_id=d32cb4e506d5426cb05fb9c1b5702ab0, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.268316 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d02573-24d3-4d51-a73b-fa802415ad62 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.274615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-7dca94a3-59a8-4f48-a543-ee05fd034c73 None None] MFA Rules not processed for user `4a4d8a2066094a56afbae522f0629473`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:52.275929 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-7dca94a3-59a8-4f48-a543-ee05fd034c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:52.281159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 809/1589] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:52:52 2022] PUT /identity/v3/projects/dbcdc99798934d2e94b52d1806d49841/users/d32cb4e506d5426cb05fb9c1b5702ab0/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:52.282003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d02573-24d3-4d51-a73b-fa802415ad62 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.341348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 781/1590] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:52.342050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca94a3-59a8-4f48-a543-ee05fd034c73 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.354745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d02573-24d3-4d51-a73b-fa802415ad62 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.359843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5ac5d-c6bd-4bd9-ae02-edf73e8a80dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.360464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5ac5d-c6bd-4bd9-ae02-edf73e8a80dd admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.360747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5ac5d-c6bd-4bd9-ae02-edf73e8a80dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.361018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5ac5d-c6bd-4bd9-ae02-edf73e8a80dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.361714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5ac5d-c6bd-4bd9-ae02-edf73e8a80dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.363384 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5ac5d-c6bd-4bd9-ae02-edf73e8a80dd admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.373215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 810/1591] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:52.376652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5ac5d-c6bd-4bd9-ae02-edf73e8a80dd admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.383212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5ac5d-c6bd-4bd9-ae02-edf73e8a80dd admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.387769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a7b12-2a8b-4670-b2ff-958ecad13089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.388345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a7b12-2a8b-4670-b2ff-958ecad13089 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.388623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a7b12-2a8b-4670-b2ff-958ecad13089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.389156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084a7b12-2a8b-4670-b2ff-958ecad13089 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.389567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a7b12-2a8b-4670-b2ff-958ecad13089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.391176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084a7b12-2a8b-4670-b2ff-958ecad13089 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.401142 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 811/1592] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:52.401903 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a7b12-2a8b-4670-b2ff-958ecad13089 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.410408 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca94a3-59a8-4f48-a543-ee05fd034c73 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.411783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084a7b12-2a8b-4670-b2ff-958ecad13089 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.415613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d93655-9b91-4d5d-9e21-a82d907ff94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.416235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d93655-9b91-4d5d-9e21-a82d907ff94a admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.416347 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53263e91-e8af-4f5f-8206-0cc2e38f649b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.416515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d93655-9b91-4d5d-9e21-a82d907ff94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.416787 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d93655-9b91-4d5d-9e21-a82d907ff94a admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.416927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53263e91-e8af-4f5f-8206-0cc2e38f649b admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.417205 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53263e91-e8af-4f5f-8206-0cc2e38f649b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.417477 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53263e91-e8af-4f5f-8206-0cc2e38f649b admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.417586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d93655-9b91-4d5d-9e21-a82d907ff94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.418155 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53263e91-e8af-4f5f-8206-0cc2e38f649b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.419228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d93655-9b91-4d5d-9e21-a82d907ff94a admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.419787 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53263e91-e8af-4f5f-8206-0cc2e38f649b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.420765 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:52.435240 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 812/1593] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:52.435240 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53263e91-e8af-4f5f-8206-0cc2e38f649b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.435674 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 782/1594] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:52.436508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d93655-9b91-4d5d-9e21-a82d907ff94a admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.445229 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53263e91-e8af-4f5f-8206-0cc2e38f649b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.447306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d93655-9b91-4d5d-9e21-a82d907ff94a admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.449794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090a4f1-9235-4787-b73b-cdf0edf3c789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.450441 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090a4f1-9235-4787-b73b-cdf0edf3c789 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.450751 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090a4f1-9235-4787-b73b-cdf0edf3c789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.451026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090a4f1-9235-4787-b73b-cdf0edf3c789 admin admin] PATH_INFO: `/v3/projects/6f5ed78b61e1440da93d5386d4d329f9/users/8aad259ca09340fb99c6b81b4f096dc5/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.452117 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a7c3c-d1d1-4d5a-9f44-ed7f0a84d6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.452787 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a7c3c-d1d1-4d5a-9f44-ed7f0a84d6fc admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.453148 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a7c3c-d1d1-4d5a-9f44-ed7f0a84d6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.453493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a7c3c-d1d1-4d5a-9f44-ed7f0a84d6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.454260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a7c3c-d1d1-4d5a-9f44-ed7f0a84d6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.455896 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a7c3c-d1d1-4d5a-9f44-ed7f0a84d6fc admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.465083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090a4f1-9235-4787-b73b-cdf0edf3c789 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5ed78b61e1440da93d5386d4d329f9, user_id=8aad259ca09340fb99c6b81b4f096dc5, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.465633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 783/1595] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:52.466842 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a732df5-0735-4ecd-9df4-e1b8cd420caf None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.467129 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a732df5-0735-4ecd-9df4-e1b8cd420caf None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.467405 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a732df5-0735-4ecd-9df4-e1b8cd420caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.467739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090a4f1-9235-4787-b73b-cdf0edf3c789 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.480619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 813/1596] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:52:52 2022] PUT /identity/v3/projects/6f5ed78b61e1440da93d5386d4d329f9/users/8aad259ca09340fb99c6b81b4f096dc5/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:52.484900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090a4f1-9235-4787-b73b-cdf0edf3c789 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.495443 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-6a732df5-0735-4ecd-9df4-e1b8cd420caf None None] MFA Rules not processed for user `d32cb4e506d5426cb05fb9c1b5702ab0`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:52.496652 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6a732df5-0735-4ecd-9df4-e1b8cd420caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:52.538754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090a4f1-9235-4787-b73b-cdf0edf3c789 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.542338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972297da-9eb3-47a2-a494-c39dd2bad742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.542987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972297da-9eb3-47a2-a494-c39dd2bad742 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.543267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972297da-9eb3-47a2-a494-c39dd2bad742 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.543539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972297da-9eb3-47a2-a494-c39dd2bad742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.552604 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:52.560903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 784/1597] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:52 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:52.561800 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a732df5-0735-4ecd-9df4-e1b8cd420caf None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.603755 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972297da-9eb3-47a2-a494-c39dd2bad742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.605180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972297da-9eb3-47a2-a494-c39dd2bad742 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.614883 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 814/1598] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:52:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:52:52.615780 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972297da-9eb3-47a2-a494-c39dd2bad742 service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.623603 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a732df5-0735-4ecd-9df4-e1b8cd420caf None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.626759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972297da-9eb3-47a2-a494-c39dd2bad742 service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.628195 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f30090-c160-4fcd-97f2-2c8c085a4903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.629501 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f30090-c160-4fcd-97f2-2c8c085a4903 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.629788 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f30090-c160-4fcd-97f2-2c8c085a4903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.630055 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f30090-c160-4fcd-97f2-2c8c085a4903 admin admin] PATH_INFO: `/v3/projects/983ea94ea58e41ccbbab9bebe0e89e3c/users/44fbd50cb5664149814abf0e9e44e18f/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.632539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedc67e-c9d1-4681-8a94-6ddd463d354d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.633152 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedc67e-c9d1-4681-8a94-6ddd463d354d admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.633428 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedc67e-c9d1-4681-8a94-6ddd463d354d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.633699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aedc67e-c9d1-4681-8a94-6ddd463d354d admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.634397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedc67e-c9d1-4681-8a94-6ddd463d354d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.636601 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aedc67e-c9d1-4681-8a94-6ddd463d354d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.645850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f30090-c160-4fcd-97f2-2c8c085a4903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983ea94ea58e41ccbbab9bebe0e89e3c, user_id=44fbd50cb5664149814abf0e9e44e18f, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.646913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: WARNING py.warnings [None req-a5f30090-c160-4fcd-97f2-2c8c085a4903 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 12:52:52.646913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: warnings.warn(msg) Aug 03 12:52:52.646913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: Aug 03 12:52:52.648478 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 815/1599] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:52.649120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f30090-c160-4fcd-97f2-2c8c085a4903 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.649224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedc67e-c9d1-4681-8a94-6ddd463d354d admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.660320 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aedc67e-c9d1-4681-8a94-6ddd463d354d admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.668148 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 785/1600] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:52:52 2022] PUT /identity/v3/projects/983ea94ea58e41ccbbab9bebe0e89e3c/users/44fbd50cb5664149814abf0e9e44e18f/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:52.668148 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f30090-c160-4fcd-97f2-2c8c085a4903 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.728366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033be32-39da-432a-84a6-1610ebf7ad80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.729159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033be32-39da-432a-84a6-1610ebf7ad80 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.729745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033be32-39da-432a-84a6-1610ebf7ad80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.729745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033be32-39da-432a-84a6-1610ebf7ad80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.730320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:52.730442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033be32-39da-432a-84a6-1610ebf7ad80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.732392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033be32-39da-432a-84a6-1610ebf7ad80 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.734605 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f30090-c160-4fcd-97f2-2c8c085a4903 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.740912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8027b-c9aa-40f9-aa8e-9e0efb8c378d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.741700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8027b-c9aa-40f9-aa8e-9e0efb8c378d admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.742078 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8027b-c9aa-40f9-aa8e-9e0efb8c378d admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.742429 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa8027b-c9aa-40f9-aa8e-9e0efb8c378d admin admin] PATH_INFO: `/v3/projects/6a201670d3cc4449852eff42aafea387/users/73ddc764dd0b41a0a1b1d8a67ed551ca/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.743840 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 816/1601] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:52.744644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033be32-39da-432a-84a6-1610ebf7ad80 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.755013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033be32-39da-432a-84a6-1610ebf7ad80 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.759720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb72eef-cc6b-4f5e-87c5-67acc9a2c903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.760328 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb72eef-cc6b-4f5e-87c5-67acc9a2c903 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.760606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb72eef-cc6b-4f5e-87c5-67acc9a2c903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.760877 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb72eef-cc6b-4f5e-87c5-67acc9a2c903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.762060 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb72eef-cc6b-4f5e-87c5-67acc9a2c903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.763683 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb72eef-cc6b-4f5e-87c5-67acc9a2c903 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.764086 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8027b-c9aa-40f9-aa8e-9e0efb8c378d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a201670d3cc4449852eff42aafea387, user_id=73ddc764dd0b41a0a1b1d8a67ed551ca, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.766834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa8027b-c9aa-40f9-aa8e-9e0efb8c378d admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.776556 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 817/1602] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:52.778061 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c4a9b-03c8-44c9-9341-4613f1df4754 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.778503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c4a9b-03c8-44c9-9341-4613f1df4754 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.778939 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738c4a9b-03c8-44c9-9341-4613f1df4754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.782642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 786/1603] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:52:52 2022] PUT /identity/v3/projects/6a201670d3cc4449852eff42aafea387/users/73ddc764dd0b41a0a1b1d8a67ed551ca/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:52.783452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8027b-c9aa-40f9-aa8e-9e0efb8c378d admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.802871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-738c4a9b-03c8-44c9-9341-4613f1df4754 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:52.804005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-738c4a9b-03c8-44c9-9341-4613f1df4754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:52.854616 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa8027b-c9aa-40f9-aa8e-9e0efb8c378d admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.864183 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321ab56-f1ff-4d65-b560-0ed76316a8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.864823 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321ab56-f1ff-4d65-b560-0ed76316a8c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.865109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321ab56-f1ff-4d65-b560-0ed76316a8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.865421 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321ab56-f1ff-4d65-b560-0ed76316a8c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.866094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321ab56-f1ff-4d65-b560-0ed76316a8c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.868957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321ab56-f1ff-4d65-b560-0ed76316a8c6 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.869693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 818/1604] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:52.870643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c4a9b-03c8-44c9-9341-4613f1df4754 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.881104 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738c4a9b-03c8-44c9-9341-4613f1df4754 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:52.886111 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c9f0e-8737-480a-9613-bfaf25f35221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:52.886820 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c9f0e-8737-480a-9613-bfaf25f35221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.887157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c9f0e-8737-480a-9613-bfaf25f35221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.887427 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c9f0e-8737-480a-9613-bfaf25f35221 admin admin] PATH_INFO: `/v3/projects/d5e359c6007d4781972a3c09d817e2bd/users/5f16ff28aac54514a475f8a4f5c6e4c4/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.903191 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c9f0e-8737-480a-9613-bfaf25f35221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5e359c6007d4781972a3c09d817e2bd, user_id=5f16ff28aac54514a475f8a4f5c6e4c4, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:52.905848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c9f0e-8737-480a-9613-bfaf25f35221 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:52.909554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 787/1605] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:52:52 2022] POST /identity/v3/projects => generated 399 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:52.909554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe794f0f-6138-4aae-99c5-bef676312353 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:52.909554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe794f0f-6138-4aae-99c5-bef676312353 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:52.909554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe794f0f-6138-4aae-99c5-bef676312353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:52.917716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:52.923791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 819/1606] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:52:52 2022] PUT /identity/v3/projects/d5e359c6007d4781972a3c09d817e2bd/users/5f16ff28aac54514a475f8a4f5c6e4c4/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:52.929604 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c9f0e-8737-480a-9613-bfaf25f35221 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:52.934925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-fe794f0f-6138-4aae-99c5-bef676312353 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:52.936273 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-fe794f0f-6138-4aae-99c5-bef676312353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:52.997161 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c9f0e-8737-480a-9613-bfaf25f35221 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.001842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82be3c2-35f2-4dc1-934a-3cf82447b195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.002528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82be3c2-35f2-4dc1-934a-3cf82447b195 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.002810 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82be3c2-35f2-4dc1-934a-3cf82447b195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.003091 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82be3c2-35f2-4dc1-934a-3cf82447b195 admin admin] PATH_INFO: `/v3/projects/1608d8997d504543a9cae8e8307581d5/users/13128464bb2547f5838a787d022e6c5f/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.005441 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 788/1607] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:53.006246 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe794f0f-6138-4aae-99c5-bef676312353 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:53.020929 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe794f0f-6138-4aae-99c5-bef676312353 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.023882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82be3c2-35f2-4dc1-934a-3cf82447b195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1608d8997d504543a9cae8e8307581d5, user_id=13128464bb2547f5838a787d022e6c5f, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:53.026141 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b570d-53c5-49d3-be60-1449175a92ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.026671 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82be3c2-35f2-4dc1-934a-3cf82447b195 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:53.027033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b570d-53c5-49d3-be60-1449175a92ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.027399 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b570d-53c5-49d3-be60-1449175a92ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.027737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b570d-53c5-49d3-be60-1449175a92ac admin admin] PATH_INFO: `/v3/projects/db5a68fd5cdb47ec96f455c518d99919/users/224194e69d0b4d02b1a5b8c3913aa7cf/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.048841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 820/1608] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:52:52 2022] PUT /identity/v3/projects/1608d8997d504543a9cae8e8307581d5/users/13128464bb2547f5838a787d022e6c5f/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:53.050963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3c8ab-256d-4d19-b6f4-8f84af8a68c3 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.051306 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3c8ab-256d-4d19-b6f4-8f84af8a68c3 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.051587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3c8ab-256d-4d19-b6f4-8f84af8a68c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.054001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b570d-53c5-49d3-be60-1449175a92ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=db5a68fd5cdb47ec96f455c518d99919, user_id=224194e69d0b4d02b1a5b8c3913aa7cf, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:53.057397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b570d-53c5-49d3-be60-1449175a92ac admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:53.073578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 789/1609] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:52:53 2022] PUT /identity/v3/projects/db5a68fd5cdb47ec96f455c518d99919/users/224194e69d0b4d02b1a5b8c3913aa7cf/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:53.073799 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b570d-53c5-49d3-be60-1449175a92ac admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:53.087104 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-73a3c8ab-256d-4d19-b6f4-8f84af8a68c3 None None] MFA Rules not processed for user `73ddc764dd0b41a0a1b1d8a67ed551ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:53.088385 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-73a3c8ab-256d-4d19-b6f4-8f84af8a68c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:53.132888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:53.146294 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b570d-53c5-49d3-be60-1449175a92ac admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.151741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b5a7b-d44d-4aeb-bcaa-bc09e6ab5abd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.152561 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b5a7b-d44d-4aeb-bcaa-bc09e6ab5abd service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.152948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b5a7b-d44d-4aeb-bcaa-bc09e6ab5abd service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.153291 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b5a7b-d44d-4aeb-bcaa-bc09e6ab5abd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.171879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 821/1610] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:53.172947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3c8ab-256d-4d19-b6f4-8f84af8a68c3 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:53.236745 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b5a7b-d44d-4aeb-bcaa-bc09e6ab5abd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:53.238711 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b5a7b-d44d-4aeb-bcaa-bc09e6ab5abd service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:53.256487 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 790/1611] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:52:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:52:53.257858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f44f8c-40a4-410b-b7a0-4f5fa833cfee None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.258275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f44f8c-40a4-410b-b7a0-4f5fa833cfee None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.258663 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f44f8c-40a4-410b-b7a0-4f5fa833cfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.261204 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3c8ab-256d-4d19-b6f4-8f84af8a68c3 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.266882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40604508-d2aa-4769-b5b1-1c6e6ab461b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.268700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40604508-d2aa-4769-b5b1-1c6e6ab461b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.269584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40604508-d2aa-4769-b5b1-1c6e6ab461b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.269932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40604508-d2aa-4769-b5b1-1c6e6ab461b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.270708 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40604508-d2aa-4769-b5b1-1c6e6ab461b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:53.273068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40604508-d2aa-4769-b5b1-1c6e6ab461b8 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:53.280535 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-e4f44f8c-40a4-410b-b7a0-4f5fa833cfee None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:53.281839 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e4f44f8c-40a4-410b-b7a0-4f5fa833cfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:53.291893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 791/1612] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:53.300641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f44f8c-40a4-410b-b7a0-4f5fa833cfee None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:53.304614 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 822/1613] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:52:53 2022] POST /identity/v3/users => generated 306 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:53.306537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceded1ef-a4c1-4c00-bb7e-17ba59a0d564 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.306950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceded1ef-a4c1-4c00-bb7e-17ba59a0d564 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.308056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceded1ef-a4c1-4c00-bb7e-17ba59a0d564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.312076 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f44f8c-40a4-410b-b7a0-4f5fa833cfee None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.316122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e4bc0-4c15-4f09-9994-866186a81a00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.316759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e4bc0-4c15-4f09-9994-866186a81a00 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.317112 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e4bc0-4c15-4f09-9994-866186a81a00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.317249 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:53.317516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e4bc0-4c15-4f09-9994-866186a81a00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.333122 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-ceded1ef-a4c1-4c00-bb7e-17ba59a0d564 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:53.334837 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ceded1ef-a4c1-4c00-bb7e-17ba59a0d564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:53.335621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e4bc0-4c15-4f09-9994-866186a81a00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:53.337069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738e4bc0-4c15-4f09-9994-866186a81a00 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:53.346383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 823/1614] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:53.348643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cde7c-c99e-49f2-983d-0b23f7b86f13 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.349068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cde7c-c99e-49f2-983d-0b23f7b86f13 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.349320 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1cde7c-c99e-49f2-983d-0b23f7b86f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.350297 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 792/1615] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:52:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:52:53.351215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e4bc0-4c15-4f09-9994-866186a81a00 service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:53.364784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738e4bc0-4c15-4f09-9994-866186a81a00 service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.373397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5d6e5-aa0e-470b-a982-d7f5e0b11c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.374622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5d6e5-aa0e-470b-a982-d7f5e0b11c7c admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.375037 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5d6e5-aa0e-470b-a982-d7f5e0b11c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.375157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5d6e5-aa0e-470b-a982-d7f5e0b11c7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.375818 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5d6e5-aa0e-470b-a982-d7f5e0b11c7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:53.377848 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5d6e5-aa0e-470b-a982-d7f5e0b11c7c admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:53.385739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-3e1cde7c-c99e-49f2-983d-0b23f7b86f13 None None] MFA Rules not processed for user `224194e69d0b4d02b1a5b8c3913aa7cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:53.387161 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-3e1cde7c-c99e-49f2-983d-0b23f7b86f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:53.412368 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 793/1616] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:52:53 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:53.413273 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5d6e5-aa0e-470b-a982-d7f5e0b11c7c admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:53.476660 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 824/1617] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:53.477658 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1cde7c-c99e-49f2-983d-0b23f7b86f13 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:53.486886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5d6e5-aa0e-470b-a982-d7f5e0b11c7c admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.491354 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d042fd-7f72-46d3-b41f-3b586718fe67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.491988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d042fd-7f72-46d3-b41f-3b586718fe67 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.492342 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d042fd-7f72-46d3-b41f-3b586718fe67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.492549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d042fd-7f72-46d3-b41f-3b586718fe67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.492692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1cde7c-c99e-49f2-983d-0b23f7b86f13 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.560871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de52839-756c-4744-b84a-ee0a45cad4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.562100 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de52839-756c-4744-b84a-ee0a45cad4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.562591 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de52839-756c-4744-b84a-ee0a45cad4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.562958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de52839-756c-4744-b84a-ee0a45cad4d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.563563 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de52839-756c-4744-b84a-ee0a45cad4d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:53.564972 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d042fd-7f72-46d3-b41f-3b586718fe67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:53.565551 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de52839-756c-4744-b84a-ee0a45cad4d5 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:53.568718 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d042fd-7f72-46d3-b41f-3b586718fe67 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:53.576603 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 825/1618] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:53.577013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:53.577431 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de52839-756c-4744-b84a-ee0a45cad4d5 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:53.589791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de52839-756c-4744-b84a-ee0a45cad4d5 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.591424 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 794/1619] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:52:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:52:53.592293 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d042fd-7f72-46d3-b41f-3b586718fe67 service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:53.598956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadaf12f-38c2-4cc2-84f2-52c5360ec812 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.599817 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadaf12f-38c2-4cc2-84f2-52c5360ec812 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.600072 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadaf12f-38c2-4cc2-84f2-52c5360ec812 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.600310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadaf12f-38c2-4cc2-84f2-52c5360ec812 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.607958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d042fd-7f72-46d3-b41f-3b586718fe67 service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.617499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadaf12f-38c2-4cc2-84f2-52c5360ec812 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:53.618991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadaf12f-38c2-4cc2-84f2-52c5360ec812 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:53.621556 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17c12f-52be-4bb1-b8d5-06326164f38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.622359 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17c12f-52be-4bb1-b8d5-06326164f38b admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.623239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17c12f-52be-4bb1-b8d5-06326164f38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.623239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17c12f-52be-4bb1-b8d5-06326164f38b admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.624094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17c12f-52be-4bb1-b8d5-06326164f38b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:53.626904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17c12f-52be-4bb1-b8d5-06326164f38b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:53.631578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 826/1620] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:52:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:52:53.632612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadaf12f-38c2-4cc2-84f2-52c5360ec812 service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:53.644403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadaf12f-38c2-4cc2-84f2-52c5360ec812 service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.650810 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56eefa-23aa-42b0-ac59-f76e63cfc9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.651641 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56eefa-23aa-42b0-ac59-f76e63cfc9b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.652232 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56eefa-23aa-42b0-ac59-f76e63cfc9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.652662 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56eefa-23aa-42b0-ac59-f76e63cfc9b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.653610 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56eefa-23aa-42b0-ac59-f76e63cfc9b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:53.655699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56eefa-23aa-42b0-ac59-f76e63cfc9b0 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:53.660836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 795/1621] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:52:53 2022] POST /identity/v3/users => generated 293 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:53.660836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17c12f-52be-4bb1-b8d5-06326164f38b admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:53.672456 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17c12f-52be-4bb1-b8d5-06326164f38b admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.702058 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c914c0-f71a-42fd-8d3a-33e92dd817b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.704045 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c914c0-f71a-42fd-8d3a-33e92dd817b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.704498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 827/1622] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:52:53 2022] POST /identity/v3/projects => generated 365 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:53.705589 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56eefa-23aa-42b0-ac59-f76e63cfc9b0 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:53.705726 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c914c0-f71a-42fd-8d3a-33e92dd817b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.706687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c914c0-f71a-42fd-8d3a-33e92dd817b0 admin admin] PATH_INFO: `/v3/projects/44f7ba2211324ca780bf2eb2c440e925/users/db1a3706e9b94634b24684ce1e1e2fee/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.732570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c914c0-f71a-42fd-8d3a-33e92dd817b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f7ba2211324ca780bf2eb2c440e925, user_id=db1a3706e9b94634b24684ce1e1e2fee, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:53.737002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c914c0-f71a-42fd-8d3a-33e92dd817b0 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:53.769529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 796/1623] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:52:53 2022] PUT /identity/v3/projects/44f7ba2211324ca780bf2eb2c440e925/users/db1a3706e9b94634b24684ce1e1e2fee/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:53.773100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c914c0-f71a-42fd-8d3a-33e92dd817b0 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:53.778665 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:53.819582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56eefa-23aa-42b0-ac59-f76e63cfc9b0 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.887327 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c914c0-f71a-42fd-8d3a-33e92dd817b0 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.893495 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f097603-554a-49f7-b002-c08f39bafa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.893874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f097603-554a-49f7-b002-c08f39bafa21 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.894282 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f097603-554a-49f7-b002-c08f39bafa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.894692 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f097603-554a-49f7-b002-c08f39bafa21 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.895441 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f097603-554a-49f7-b002-c08f39bafa21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:53.897918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f097603-554a-49f7-b002-c08f39bafa21 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:53.922144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb53d8-2fe0-46a1-a01c-e3b3a7834b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.922938 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb53d8-2fe0-46a1-a01c-e3b3a7834b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.923219 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb53d8-2fe0-46a1-a01c-e3b3a7834b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.924089 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb53d8-2fe0-46a1-a01c-e3b3a7834b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.928347 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb53d8-2fe0-46a1-a01c-e3b3a7834b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:53.928518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 797/1624] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:52:53 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:52:53.928609 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb53d8-2fe0-46a1-a01c-e3b3a7834b18 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:53.928703 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f097603-554a-49f7-b002-c08f39bafa21 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:53.944982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f097603-554a-49f7-b002-c08f39bafa21 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.946353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 828/1625] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:53.947252 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb53d8-2fe0-46a1-a01c-e3b3a7834b18 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:53.953258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ddea0-46e6-4548-ad7b-aac8935a8892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.954571 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ddea0-46e6-4548-ad7b-aac8935a8892 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.961754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb53d8-2fe0-46a1-a01c-e3b3a7834b18 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.963424 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ddea0-46e6-4548-ad7b-aac8935a8892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.963766 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ddea0-46e6-4548-ad7b-aac8935a8892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.964601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ddea0-46e6-4548-ad7b-aac8935a8892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:53.966321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ddea0-46e6-4548-ad7b-aac8935a8892 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:53.978637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 798/1626] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:53.979528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ddea0-46e6-4548-ad7b-aac8935a8892 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:53.991813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56edbc-f58a-4265-ae0e-c39de44932e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.992446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ddea0-46e6-4548-ad7b-aac8935a8892 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:53.992595 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56edbc-f58a-4265-ae0e-c39de44932e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.992804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56edbc-f58a-4265-ae0e-c39de44932e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.993015 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56edbc-f58a-4265-ae0e-c39de44932e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:53.993739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56edbc-f58a-4265-ae0e-c39de44932e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:53.996048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56edbc-f58a-4265-ae0e-c39de44932e8 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:53.997945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1956cfd-c9b0-4138-a9a6-16d1ad4f007c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:53.998598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1956cfd-c9b0-4138-a9a6-16d1ad4f007c admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:53.998886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1956cfd-c9b0-4138-a9a6-16d1ad4f007c admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:53.999164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1956cfd-c9b0-4138-a9a6-16d1ad4f007c admin admin] PATH_INFO: `/v3/projects/525d62607de94c819c18ef993aa97a80/users/9ba79d3c286b49c08715dbc7982f6b70/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:54.025314 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 829/1627] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:54.025314 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56edbc-f58a-4265-ae0e-c39de44932e8 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:54.030721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1956cfd-c9b0-4138-a9a6-16d1ad4f007c admin admin] RBAC: Authorizing `identity:create_grant(project_id=525d62607de94c819c18ef993aa97a80, user_id=9ba79d3c286b49c08715dbc7982f6b70, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:54.033577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1956cfd-c9b0-4138-a9a6-16d1ad4f007c admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:54.045601 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56edbc-f58a-4265-ae0e-c39de44932e8 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:54.057154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828160ed-dcc9-4e41-acda-4a3e04ca2502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:54.058581 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828160ed-dcc9-4e41-acda-4a3e04ca2502 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:54.059031 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828160ed-dcc9-4e41-acda-4a3e04ca2502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:54.059842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828160ed-dcc9-4e41-acda-4a3e04ca2502 admin admin] PATH_INFO: `/v3/projects/44f7ba2211324ca780bf2eb2c440e925/users/db1a3706e9b94634b24684ce1e1e2fee/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:54.062416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828160ed-dcc9-4e41-acda-4a3e04ca2502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f7ba2211324ca780bf2eb2c440e925, user_id=db1a3706e9b94634b24684ce1e1e2fee, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:54.065387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 799/1628] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:52:53 2022] PUT /identity/v3/projects/525d62607de94c819c18ef993aa97a80/users/9ba79d3c286b49c08715dbc7982f6b70/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:54.065387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1956cfd-c9b0-4138-a9a6-16d1ad4f007c admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:54.070318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828160ed-dcc9-4e41-acda-4a3e04ca2502 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:54.098647 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 830/1629] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:52:54 2022] PUT /identity/v3/projects/44f7ba2211324ca780bf2eb2c440e925/users/db1a3706e9b94634b24684ce1e1e2fee/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:54.100547 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828160ed-dcc9-4e41-acda-4a3e04ca2502 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:54.169808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1956cfd-c9b0-4138-a9a6-16d1ad4f007c admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:54.199476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:54.238906 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828160ed-dcc9-4e41-acda-4a3e04ca2502 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:54.245079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad64141-9abd-42ac-b10a-8709df867098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:54.245719 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad64141-9abd-42ac-b10a-8709df867098 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:54.246005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad64141-9abd-42ac-b10a-8709df867098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:54.246282 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad64141-9abd-42ac-b10a-8709df867098 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:54.247433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad64141-9abd-42ac-b10a-8709df867098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:54.249479 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad64141-9abd-42ac-b10a-8709df867098 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:54.266605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 831/1630] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:54.268039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77d58c-1aec-45e4-a3c7-15c598169c1d None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:54.268328 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77d58c-1aec-45e4-a3c7-15c598169c1d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:54.268608 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77d58c-1aec-45e4-a3c7-15c598169c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:54.269537 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d60b5-d291-4fc7-8571-3e7f29903436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:54.270680 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d60b5-d291-4fc7-8571-3e7f29903436 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:54.271190 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d60b5-d291-4fc7-8571-3e7f29903436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:54.271679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d60b5-d291-4fc7-8571-3e7f29903436 admin admin] PATH_INFO: `/v3/projects/274c9d886a614ac9928f2b43c8d83d62/users/0b5e1c98cccc406f82d4f02c97037841/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:54.296506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d60b5-d291-4fc7-8571-3e7f29903436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=274c9d886a614ac9928f2b43c8d83d62, user_id=0b5e1c98cccc406f82d4f02c97037841, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:54.300246 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d60b5-d291-4fc7-8571-3e7f29903436 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:54.311215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1d77d58c-1aec-45e4-a3c7-15c598169c1d None None] MFA Rules not processed for user `db1a3706e9b94634b24684ce1e1e2fee`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:54.312794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1d77d58c-1aec-45e4-a3c7-15c598169c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:54.317639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 800/1631] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:52:54 2022] PUT /identity/v3/projects/274c9d886a614ac9928f2b43c8d83d62/users/0b5e1c98cccc406f82d4f02c97037841/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:54.318781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d60b5-d291-4fc7-8571-3e7f29903436 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:54.437708 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:54.452618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d60b5-d291-4fc7-8571-3e7f29903436 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:54.466223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da13e0-489b-4512-8670-d382d9469cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:54.467804 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da13e0-489b-4512-8670-d382d9469cc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:54.468049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da13e0-489b-4512-8670-d382d9469cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:54.471813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da13e0-489b-4512-8670-d382d9469cc0 admin admin] PATH_INFO: `/v3/projects/525d62607de94c819c18ef993aa97a80/users/9ba79d3c286b49c08715dbc7982f6b70/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:54.471961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 832/1632] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:54 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:54.471961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77d58c-1aec-45e4-a3c7-15c598169c1d None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:54.472674 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0da13e0-489b-4512-8670-d382d9469cc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=525d62607de94c819c18ef993aa97a80, user_id=9ba79d3c286b49c08715dbc7982f6b70, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:54.475411 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0da13e0-489b-4512-8670-d382d9469cc0 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:54.503197 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77d58c-1aec-45e4-a3c7-15c598169c1d None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:54.505851 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 801/1633] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:52:54 2022] PUT /identity/v3/projects/525d62607de94c819c18ef993aa97a80/users/9ba79d3c286b49c08715dbc7982f6b70/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:54.521297 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da13e0-489b-4512-8670-d382d9469cc0 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:54.576627 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c41e6-5d31-42bc-afd8-85a6cef5fff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:54.577309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c41e6-5d31-42bc-afd8-85a6cef5fff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:54.577595 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c41e6-5d31-42bc-afd8-85a6cef5fff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:54.577869 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c41e6-5d31-42bc-afd8-85a6cef5fff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:54.578980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c41e6-5d31-42bc-afd8-85a6cef5fff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:54.580747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021c41e6-5d31-42bc-afd8-85a6cef5fff2 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:54.597938 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 833/1634] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:52:54 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:52:54.601996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f7030-fb5c-4284-a545-1502f88240bd None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:54.602980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f7030-fb5c-4284-a545-1502f88240bd None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:54.603270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f7030-fb5c-4284-a545-1502f88240bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:54.617594 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da13e0-489b-4512-8670-d382d9469cc0 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:54.623358 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb0ea3-de21-409a-badb-d5830b230496 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:54.629502 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb0ea3-de21-409a-badb-d5830b230496 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:54.629846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb0ea3-de21-409a-badb-d5830b230496 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:54.635693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb0ea3-de21-409a-badb-d5830b230496 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:54.638397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-3e5f7030-fb5c-4284-a545-1502f88240bd None None] MFA Rules not processed for user `9ba79d3c286b49c08715dbc7982f6b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:54.639662 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-3e5f7030-fb5c-4284-a545-1502f88240bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:54.706347 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:54.727097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb0ea3-de21-409a-badb-d5830b230496 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:54.729293 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb0ea3-de21-409a-badb-d5830b230496 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:54.733057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 834/1635] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:54.734077 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f7030-fb5c-4284-a545-1502f88240bd None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:54.755970 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 802/1636] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:52:54 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:52:54.757120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5f7030-fb5c-4284-a545-1502f88240bd None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:54.757996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb0ea3-de21-409a-badb-d5830b230496 service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:54.764230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26c3f5-1a79-4093-b76e-52703565dd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:54.765791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26c3f5-1a79-4093-b76e-52703565dd4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:54.765791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26c3f5-1a79-4093-b76e-52703565dd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:54.766051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26c3f5-1a79-4093-b76e-52703565dd4f admin admin] PATH_INFO: `/v3/projects/274c9d886a614ac9928f2b43c8d83d62/users/0b5e1c98cccc406f82d4f02c97037841/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:54.768714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26c3f5-1a79-4093-b76e-52703565dd4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=274c9d886a614ac9928f2b43c8d83d62, user_id=0b5e1c98cccc406f82d4f02c97037841, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:54.770894 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb0ea3-de21-409a-badb-d5830b230496 service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:54.772653 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26c3f5-1a79-4093-b76e-52703565dd4f admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:54.777873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826ce12-9fe3-49be-976c-5a74624a0347 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:54.778955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826ce12-9fe3-49be-976c-5a74624a0347 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:54.796089 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 835/1637] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:52:54 2022] PUT /identity/v3/projects/274c9d886a614ac9928f2b43c8d83d62/users/0b5e1c98cccc406f82d4f02c97037841/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:54.798641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826ce12-9fe3-49be-976c-5a74624a0347 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:54.799266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826ce12-9fe3-49be-976c-5a74624a0347 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:54.806450 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cf9b4-c7d1-4862-8a0b-f3f5d9dd5bd1 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:54.806758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cf9b4-c7d1-4862-8a0b-f3f5d9dd5bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:54.807037 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768cf9b4-c7d1-4862-8a0b-f3f5d9dd5bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:54.820672 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826ce12-9fe3-49be-976c-5a74624a0347 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:54.823544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2826ce12-9fe3-49be-976c-5a74624a0347 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:54.844763 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 803/1638] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:52:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:52:54.890508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c259c8-f6b5-4a45-b9f0-4c778bf89005 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:54.890855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c259c8-f6b5-4a45-b9f0-4c778bf89005 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:54.891129 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c259c8-f6b5-4a45-b9f0-4c778bf89005 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:54.893007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 804/1639] 10.222.0.65 () {54 vars in 962 bytes} [Wed Aug 3 12:52:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:52:54.903447 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f7ed2-c14e-4d44-8dd2-f42d92b8faf5 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:54.903776 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f7ed2-c14e-4d44-8dd2-f42d92b8faf5 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:54.904400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533f7ed2-c14e-4d44-8dd2-f42d92b8faf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:54.917109 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-768cf9b4-c7d1-4862-8a0b-f3f5d9dd5bd1 None None] MFA Rules not processed for user `0b5e1c98cccc406f82d4f02c97037841`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:54.919027 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-768cf9b4-c7d1-4862-8a0b-f3f5d9dd5bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:55.023601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-533f7ed2-c14e-4d44-8dd2-f42d92b8faf5 None None] MFA Rules not processed for user `7b634f567dca42dc95df175a4f36c1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:55.024863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-533f7ed2-c14e-4d44-8dd2-f42d92b8faf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:55.057591 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:55.118841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 836/1640] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:55.118841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cf9b4-c7d1-4862-8a0b-f3f5d9dd5bd1 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:55.185600 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 805/1641] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Aug 3 12:52:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:52:55.187332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f7ed2-c14e-4d44-8dd2-f42d92b8faf5 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:55.264783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768cf9b4-c7d1-4862-8a0b-f3f5d9dd5bd1 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:55.276584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ce68b-a535-4b7e-8925-687d000d36d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:55.277557 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ce68b-a535-4b7e-8925-687d000d36d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:55.279808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ce68b-a535-4b7e-8925-687d000d36d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:55.281226 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ce68b-a535-4b7e-8925-687d000d36d1 admin admin] PATH_INFO: `/v3/users/d32cb4e506d5426cb05fb9c1b5702ab0` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:55.285225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ce68b-a535-4b7e-8925-687d000d36d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d32cb4e506d5426cb05fb9c1b5702ab0)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:55.287486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ce68b-a535-4b7e-8925-687d000d36d1 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:55.344394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533f7ed2-c14e-4d44-8dd2-f42d92b8faf5 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:55.350340 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280ad93-4723-4bab-a409-70226e784ab9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:55.351119 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280ad93-4723-4bab-a409-70226e784ab9 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:55.351477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280ad93-4723-4bab-a409-70226e784ab9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:55.351829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280ad93-4723-4bab-a409-70226e784ab9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:55.376227 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280ad93-4723-4bab-a409-70226e784ab9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:55.379155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280ad93-4723-4bab-a409-70226e784ab9 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:55.398477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 806/1642] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:52:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:52:55.401799 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025083bb-271b-4871-8854-1d54f26d3c36 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:55.402142 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025083bb-271b-4871-8854-1d54f26d3c36 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:55.402426 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025083bb-271b-4871-8854-1d54f26d3c36 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:55.404204 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 807/1643] 10.222.0.65 () {54 vars in 912 bytes} [Wed Aug 3 12:52:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:52:55.409575 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327529e-3028-430f-a64d-fe802f97bff5 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:55.410172 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327529e-3028-430f-a64d-fe802f97bff5 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:55.410573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327529e-3028-430f-a64d-fe802f97bff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:55.428567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a49ce68b-a535-4b7e-8925-687d000d36d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd32cb4e506d5426cb05fb9c1b5702ab0'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:55.445697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a49ce68b-a535-4b7e-8925-687d000d36d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd32cb4e506d5426cb05fb9c1b5702ab0'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:55.451860 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-0327529e-3028-430f-a64d-fe802f97bff5 None None] MFA Rules not processed for user `7b634f567dca42dc95df175a4f36c1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:55.453746 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-0327529e-3028-430f-a64d-fe802f97bff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:55.483942 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 808/1644] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 3 12:52:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:52:55.484753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a49ce68b-a535-4b7e-8925-687d000d36d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd32cb4e506d5426cb05fb9c1b5702ab0'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:55.485762 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a49ce68b-a535-4b7e-8925-687d000d36d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd32cb4e506d5426cb05fb9c1b5702ab0'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:55.486639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a49ce68b-a535-4b7e-8925-687d000d36d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd32cb4e506d5426cb05fb9c1b5702ab0'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:55.520278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 837/1645] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:52:55 2022] DELETE /identity/v3/users/d32cb4e506d5426cb05fb9c1b5702ab0 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:55.522156 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f4780-e38d-4c63-898c-c2223822f522 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:55.522300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f4780-e38d-4c63-898c-c2223822f522 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:55.522537 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f4780-e38d-4c63-898c-c2223822f522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:55.540997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ce68b-a535-4b7e-8925-687d000d36d1 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:55.542501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a49ce68b-a535-4b7e-8925-687d000d36d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:55.568753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-b06f4780-e38d-4c63-898c-c2223822f522 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:55.571862 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-b06f4780-e38d-4c63-898c-c2223822f522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:55.666394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ce68b-a535-4b7e-8925-687d000d36d1 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:55.671147 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faafc0-8783-47c4-b4a7-c2f5219934ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:55.671773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faafc0-8783-47c4-b4a7-c2f5219934ab service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:55.672262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faafc0-8783-47c4-b4a7-c2f5219934ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:55.672835 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faafc0-8783-47c4-b4a7-c2f5219934ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:55.674554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-56faafc0-8783-47c4-b4a7-c2f5219934ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:55.687383 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:55.699445 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:55.750516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 809/1646] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:55.768156 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f4780-e38d-4c63-898c-c2223822f522 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:55.770247 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-b06f4780-e38d-4c63-898c-c2223822f522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:55.840838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56faafc0-8783-47c4-b4a7-c2f5219934ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:55.843891 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56faafc0-8783-47c4-b4a7-c2f5219934ab service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:55.903124 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 838/1647] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:52:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:52:55.936737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faafc0-8783-47c4-b4a7-c2f5219934ab service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:55.938360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-56faafc0-8783-47c4-b4a7-c2f5219934ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:55.952140 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f4780-e38d-4c63-898c-c2223822f522 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:55.964281 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a009a8-a587-48e0-898c-8ef73fd39692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:55.964991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a009a8-a587-48e0-898c-8ef73fd39692 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:55.965325 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a009a8-a587-48e0-898c-8ef73fd39692 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:55.965594 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a009a8-a587-48e0-898c-8ef73fd39692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:55.966936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faafc0-8783-47c4-b4a7-c2f5219934ab service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:55.986950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1128b36-c8a1-47af-8b50-5958b10370a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:55.986950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1128b36-c8a1-47af-8b50-5958b10370a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:55.986950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1128b36-c8a1-47af-8b50-5958b10370a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:55.986950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1128b36-c8a1-47af-8b50-5958b10370a3 admin admin] PATH_INFO: `/v3/users/73ddc764dd0b41a0a1b1d8a67ed551ca` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:55.987275 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1128b36-c8a1-47af-8b50-5958b10370a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73ddc764dd0b41a0a1b1d8a67ed551ca)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:55.991043 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1128b36-c8a1-47af-8b50-5958b10370a3 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:55.993566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a009a8-a587-48e0-898c-8ef73fd39692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:55.997220 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a009a8-a587-48e0-898c-8ef73fd39692 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:56.034196 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 810/1648] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:52:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:52:56.092220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a1128b36-c8a1-47af-8b50-5958b10370a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73ddc764dd0b41a0a1b1d8a67ed551ca'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:56.104366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a1128b36-c8a1-47af-8b50-5958b10370a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73ddc764dd0b41a0a1b1d8a67ed551ca'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:56.129932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a1128b36-c8a1-47af-8b50-5958b10370a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73ddc764dd0b41a0a1b1d8a67ed551ca'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:56.130855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a1128b36-c8a1-47af-8b50-5958b10370a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73ddc764dd0b41a0a1b1d8a67ed551ca'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:56.133880 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a1128b36-c8a1-47af-8b50-5958b10370a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73ddc764dd0b41a0a1b1d8a67ed551ca'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:56.156292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 839/1649] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:52:55 2022] DELETE /identity/v3/users/73ddc764dd0b41a0a1b1d8a67ed551ca => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:56.161990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60678a42-2d42-424d-bd02-13e0a89ac262 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:56.162275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60678a42-2d42-424d-bd02-13e0a89ac262 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:56.162764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60678a42-2d42-424d-bd02-13e0a89ac262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:56.194116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-60678a42-2d42-424d-bd02-13e0a89ac262 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:56.199128 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-60678a42-2d42-424d-bd02-13e0a89ac262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:56.359946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 811/1650] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:56.399416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1128b36-c8a1-47af-8b50-5958b10370a3 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:56.404019 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a1128b36-c8a1-47af-8b50-5958b10370a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:56.522989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60678a42-2d42-424d-bd02-13e0a89ac262 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:56.524763 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-60678a42-2d42-424d-bd02-13e0a89ac262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:56.548908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60678a42-2d42-424d-bd02-13e0a89ac262 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:56.556075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e5f89-2036-4898-a2eb-ab829665cc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:56.556874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e5f89-2036-4898-a2eb-ab829665cc40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:56.557260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e5f89-2036-4898-a2eb-ab829665cc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:56.557639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e5f89-2036-4898-a2eb-ab829665cc40 admin admin] PATH_INFO: `/v3/users/224194e69d0b4d02b1a5b8c3913aa7cf` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:56.560990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e5f89-2036-4898-a2eb-ab829665cc40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=224194e69d0b4d02b1a5b8c3913aa7cf)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:56.562317 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: WARNING py.warnings [None req-926e5f89-2036-4898-a2eb-ab829665cc40 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 12:52:56.562317 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: warnings.warn(msg) Aug 03 12:52:56.562317 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: Aug 03 12:52:56.563822 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e5f89-2036-4898-a2eb-ab829665cc40 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:56.564351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1128b36-c8a1-47af-8b50-5958b10370a3 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:56.570072 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9116d105-b4fb-4721-9ece-827ec508a075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:56.571651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116d105-b4fb-4721-9ece-827ec508a075 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:56.572025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116d105-b4fb-4721-9ece-827ec508a075 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:56.573311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9116d105-b4fb-4721-9ece-827ec508a075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:56.587471 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116d105-b4fb-4721-9ece-827ec508a075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:56.590114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9116d105-b4fb-4721-9ece-827ec508a075 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:56.604810 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 840/1651] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:52:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:52:56.684336 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-926e5f89-2036-4898-a2eb-ab829665cc40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '224194e69d0b4d02b1a5b8c3913aa7cf'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:56.744057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-926e5f89-2036-4898-a2eb-ab829665cc40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '224194e69d0b4d02b1a5b8c3913aa7cf'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:56.762142 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-926e5f89-2036-4898-a2eb-ab829665cc40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '224194e69d0b4d02b1a5b8c3913aa7cf'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:56.773026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-926e5f89-2036-4898-a2eb-ab829665cc40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '224194e69d0b4d02b1a5b8c3913aa7cf'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:56.800225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-926e5f89-2036-4898-a2eb-ab829665cc40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '224194e69d0b4d02b1a5b8c3913aa7cf'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:56.818458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 812/1652] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:52:56 2022] DELETE /identity/v3/users/224194e69d0b4d02b1a5b8c3913aa7cf => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:52:56.823120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd276aee-ede3-484e-a220-9bed04ee9541 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:56.823323 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd276aee-ede3-484e-a220-9bed04ee9541 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:56.823537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd276aee-ede3-484e-a220-9bed04ee9541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:56.875797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-cd276aee-ede3-484e-a220-9bed04ee9541 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:56.879691 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-cd276aee-ede3-484e-a220-9bed04ee9541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:56.921910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:57.056132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 841/1653] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:52:57.085836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e5f89-2036-4898-a2eb-ab829665cc40 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:57.087453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-926e5f89-2036-4898-a2eb-ab829665cc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:57.094987 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:57.188940 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e5f89-2036-4898-a2eb-ab829665cc40 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:57.193645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba935c4-0eee-4b88-8994-9609ab912a8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:57.194500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba935c4-0eee-4b88-8994-9609ab912a8a service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:57.195005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba935c4-0eee-4b88-8994-9609ab912a8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:57.195383 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba935c4-0eee-4b88-8994-9609ab912a8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:57.209171 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba935c4-0eee-4b88-8994-9609ab912a8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:57.210722 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba935c4-0eee-4b88-8994-9609ab912a8a service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:57.223745 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 813/1654] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:52:57.532571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd276aee-ede3-484e-a220-9bed04ee9541 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:57.535607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-cd276aee-ede3-484e-a220-9bed04ee9541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:57.555318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd276aee-ede3-484e-a220-9bed04ee9541 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:57.567752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff337b0-7528-443c-b765-ed37171ea685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:57.568915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff337b0-7528-443c-b765-ed37171ea685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:57.569014 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff337b0-7528-443c-b765-ed37171ea685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:57.569226 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff337b0-7528-443c-b765-ed37171ea685 admin admin] PATH_INFO: `/v3/projects/dbcdc99798934d2e94b52d1806d49841` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:57.570761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff337b0-7528-443c-b765-ed37171ea685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbcdc99798934d2e94b52d1806d49841)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:57.572934 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: WARNING py.warnings [None req-3ff337b0-7528-443c-b765-ed37171ea685 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 03 12:52:57.572934 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: warnings.warn(msg) Aug 03 12:52:57.572934 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: Aug 03 12:52:57.583630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba935c4-0eee-4b88-8994-9609ab912a8a service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:57.588053 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-3ba935c4-0eee-4b88-8994-9609ab912a8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:57.598794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff337b0-7528-443c-b765-ed37171ea685 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:57.625510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba935c4-0eee-4b88-8994-9609ab912a8a service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:57.631984 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:57.633602 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:57.633994 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:57.634379 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0 admin admin] PATH_INFO: `/v3/projects/6a201670d3cc4449852eff42aafea387` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:57.636113 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a201670d3cc4449852eff42aafea387)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:57.638714 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:57.763416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-3ff337b0-7528-443c-b765-ed37171ea685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbcdc99798934d2e94b52d1806d49841', 'request_id': 'req-3ff337b0-7528-443c-b765-ed37171ea685'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:57.782737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-3ff337b0-7528-443c-b765-ed37171ea685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbcdc99798934d2e94b52d1806d49841', 'request_id': 'req-3ff337b0-7528-443c-b765-ed37171ea685'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:57.792535 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a201670d3cc4449852eff42aafea387', 'request_id': 'req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:57.803846 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-3ff337b0-7528-443c-b765-ed37171ea685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbcdc99798934d2e94b52d1806d49841', 'request_id': 'req-3ff337b0-7528-443c-b765-ed37171ea685'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:57.809024 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a201670d3cc4449852eff42aafea387', 'request_id': 'req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:57.822387 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-3ff337b0-7528-443c-b765-ed37171ea685 admin admin] The token cache is being invalidate because project dbcdc99798934d2e94b52d1806d49841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:52:57.822842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-3ff337b0-7528-443c-b765-ed37171ea685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:57.828842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-3ff337b0-7528-443c-b765-ed37171ea685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:57.835350 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a201670d3cc4449852eff42aafea387', 'request_id': 'req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:57.842105 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 842/1655] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:52:57 2022] DELETE /identity/v3/projects/dbcdc99798934d2e94b52d1806d49841 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:57.843290 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff337b0-7528-443c-b765-ed37171ea685 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:57.846544 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-3ff337b0-7528-443c-b765-ed37171ea685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:57.862165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0 admin admin] The token cache is being invalidate because project 6a201670d3cc4449852eff42aafea387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:52:57.872555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:57.877885 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:57.894972 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 814/1656] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:52:57 2022] DELETE /identity/v3/projects/6a201670d3cc4449852eff42aafea387 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:52:57.896139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:57.900755 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:57.990376 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff337b0-7528-443c-b765-ed37171ea685 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:58.205475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebea89-d5ff-44ae-b4d4-032ba8b87cd0 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:58.205475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41799175-6c75-4e22-8f5f-a5fe9ec6ff84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:58.205475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41799175-6c75-4e22-8f5f-a5fe9ec6ff84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:58.205475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41799175-6c75-4e22-8f5f-a5fe9ec6ff84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:58.206587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41799175-6c75-4e22-8f5f-a5fe9ec6ff84 admin admin] PATH_INFO: `/v3/users/9ba79d3c286b49c08715dbc7982f6b70` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:58.206587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41799175-6c75-4e22-8f5f-a5fe9ec6ff84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ba79d3c286b49c08715dbc7982f6b70)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:58.206587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41799175-6c75-4e22-8f5f-a5fe9ec6ff84 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:58.207309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:58.228174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:58.284259 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-41799175-6c75-4e22-8f5f-a5fe9ec6ff84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ba79d3c286b49c08715dbc7982f6b70'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:58.287452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab039f-d2f7-4270-8cf2-5f72135dbd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:58.287452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab039f-d2f7-4270-8cf2-5f72135dbd4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:58.287452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab039f-d2f7-4270-8cf2-5f72135dbd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:58.287452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab039f-d2f7-4270-8cf2-5f72135dbd4a admin admin] PATH_INFO: `/v3/users/db1a3706e9b94634b24684ce1e1e2fee` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:58.289065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab039f-d2f7-4270-8cf2-5f72135dbd4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=db1a3706e9b94634b24684ce1e1e2fee)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:58.289065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab039f-d2f7-4270-8cf2-5f72135dbd4a admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:58.300747 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-41799175-6c75-4e22-8f5f-a5fe9ec6ff84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ba79d3c286b49c08715dbc7982f6b70'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:58.333211 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-41799175-6c75-4e22-8f5f-a5fe9ec6ff84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ba79d3c286b49c08715dbc7982f6b70'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:58.334753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-41799175-6c75-4e22-8f5f-a5fe9ec6ff84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ba79d3c286b49c08715dbc7982f6b70'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:58.347267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-41799175-6c75-4e22-8f5f-a5fe9ec6ff84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ba79d3c286b49c08715dbc7982f6b70'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:58.360338 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 815/1657] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:52:57 2022] DELETE /identity/v3/users/9ba79d3c286b49c08715dbc7982f6b70 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:58.413017 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-08ab039f-d2f7-4270-8cf2-5f72135dbd4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db1a3706e9b94634b24684ce1e1e2fee'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:58.421992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-08ab039f-d2f7-4270-8cf2-5f72135dbd4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db1a3706e9b94634b24684ce1e1e2fee'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:58.452986 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-08ab039f-d2f7-4270-8cf2-5f72135dbd4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db1a3706e9b94634b24684ce1e1e2fee'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:58.453909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-08ab039f-d2f7-4270-8cf2-5f72135dbd4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db1a3706e9b94634b24684ce1e1e2fee'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:58.454670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-08ab039f-d2f7-4270-8cf2-5f72135dbd4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db1a3706e9b94634b24684ce1e1e2fee'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:58.489004 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 843/1658] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:52:57 2022] DELETE /identity/v3/users/db1a3706e9b94634b24684ce1e1e2fee => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:52:58.659280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632dfc4-0af5-41bb-8943-4088e960fdf5 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:58.661316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632dfc4-0af5-41bb-8943-4088e960fdf5 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:58.663505 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632dfc4-0af5-41bb-8943-4088e960fdf5 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:58.665367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 816/1659] 10.222.0.65 () {54 vars in 962 bytes} [Wed Aug 3 12:52:58 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:52:58.685179 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7b09f-127e-4368-89d9-4537c48a8b20 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:58.686375 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7b09f-127e-4368-89d9-4537c48a8b20 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:58.686788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7b09f-127e-4368-89d9-4537c48a8b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:58.745715 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-81c7b09f-127e-4368-89d9-4537c48a8b20 None None] MFA Rules not processed for user `7b634f567dca42dc95df175a4f36c1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:58.750933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-81c7b09f-127e-4368-89d9-4537c48a8b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:58.879527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d46625-729c-438c-9be0-2952ff20a31e None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:58.880102 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d46625-729c-438c-9be0-2952ff20a31e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:58.880661 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d46625-729c-438c-9be0-2952ff20a31e None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:58.884317 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 817/1660] 10.222.0.65 () {54 vars in 912 bytes} [Wed Aug 3 12:52:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:52:58.890013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d187602-2e4a-4745-a1e1-1640d50a37ab None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:58.890768 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d187602-2e4a-4745-a1e1-1640d50a37ab None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:58.891243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d187602-2e4a-4745-a1e1-1640d50a37ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:58.895995 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 844/1661] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Aug 3 12:52:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:52:58.916442 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-9d187602-2e4a-4745-a1e1-1640d50a37ab None None] MFA Rules not processed for user `7b634f567dca42dc95df175a4f36c1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:52:58.917890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-9d187602-2e4a-4745-a1e1-1640d50a37ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:58.957919 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 818/1662] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 3 12:52:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:52:58.969966 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7b09f-127e-4368-89d9-4537c48a8b20 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:58.971805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-81c7b09f-127e-4368-89d9-4537c48a8b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:59.041083 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d187602-2e4a-4745-a1e1-1640d50a37ab None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:59.043296 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-9d187602-2e4a-4745-a1e1-1640d50a37ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:52:59.119402 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c7b09f-127e-4368-89d9-4537c48a8b20 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:59.129752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009b09e-5b3d-4c7e-abcf-94cc57b798e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:59.130570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009b09e-5b3d-4c7e-abcf-94cc57b798e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:59.131082 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009b09e-5b3d-4c7e-abcf-94cc57b798e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:59.131343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009b09e-5b3d-4c7e-abcf-94cc57b798e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:59.155749 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009b09e-5b3d-4c7e-abcf-94cc57b798e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:59.160733 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009b09e-5b3d-4c7e-abcf-94cc57b798e1 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:59.175875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:52:59.191009 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 845/1663] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:52:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:52:59.258852 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d187602-2e4a-4745-a1e1-1640d50a37ab None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:59.273282 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39416a10-6e2d-4b17-aa37-6d8b4d750f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:59.274539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39416a10-6e2d-4b17-aa37-6d8b4d750f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:59.275063 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39416a10-6e2d-4b17-aa37-6d8b4d750f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:59.275358 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39416a10-6e2d-4b17-aa37-6d8b4d750f35 admin admin] PATH_INFO: `/v3/projects/db5a68fd5cdb47ec96f455c518d99919` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:59.280102 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39416a10-6e2d-4b17-aa37-6d8b4d750f35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db5a68fd5cdb47ec96f455c518d99919)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:59.280478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39416a10-6e2d-4b17-aa37-6d8b4d750f35 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:59.379598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-39416a10-6e2d-4b17-aa37-6d8b4d750f35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db5a68fd5cdb47ec96f455c518d99919', 'request_id': 'req-39416a10-6e2d-4b17-aa37-6d8b4d750f35'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:59.399219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-39416a10-6e2d-4b17-aa37-6d8b4d750f35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db5a68fd5cdb47ec96f455c518d99919', 'request_id': 'req-39416a10-6e2d-4b17-aa37-6d8b4d750f35'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:59.424216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-39416a10-6e2d-4b17-aa37-6d8b4d750f35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db5a68fd5cdb47ec96f455c518d99919', 'request_id': 'req-39416a10-6e2d-4b17-aa37-6d8b4d750f35'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:59.495049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-39416a10-6e2d-4b17-aa37-6d8b4d750f35 admin admin] The token cache is being invalidate because project db5a68fd5cdb47ec96f455c518d99919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:52:59.495789 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-39416a10-6e2d-4b17-aa37-6d8b4d750f35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:59.497657 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-39416a10-6e2d-4b17-aa37-6d8b4d750f35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:59.502778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 819/1664] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:52:59 2022] DELETE /identity/v3/projects/db5a68fd5cdb47ec96f455c518d99919 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:52:59.503726 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009b09e-5b3d-4c7e-abcf-94cc57b798e1 service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:52:59.520519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009b09e-5b3d-4c7e-abcf-94cc57b798e1 service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:52:59.712575 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210b4f2-d9cf-4094-a229-f338cb40d2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:52:59.713726 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210b4f2-d9cf-4094-a229-f338cb40d2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:52:59.714247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210b4f2-d9cf-4094-a229-f338cb40d2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:52:59.714623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210b4f2-d9cf-4094-a229-f338cb40d2a5 admin admin] PATH_INFO: `/v3/users/0b5e1c98cccc406f82d4f02c97037841` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:52:59.718222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210b4f2-d9cf-4094-a229-f338cb40d2a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b5e1c98cccc406f82d4f02c97037841)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:52:59.720293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210b4f2-d9cf-4094-a229-f338cb40d2a5 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:52:59.867641 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5210b4f2-d9cf-4094-a229-f338cb40d2a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b5e1c98cccc406f82d4f02c97037841'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:59.897550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5210b4f2-d9cf-4094-a229-f338cb40d2a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b5e1c98cccc406f82d4f02c97037841'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:59.932197 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5210b4f2-d9cf-4094-a229-f338cb40d2a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b5e1c98cccc406f82d4f02c97037841'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:59.932842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5210b4f2-d9cf-4094-a229-f338cb40d2a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b5e1c98cccc406f82d4f02c97037841'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:59.938974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5210b4f2-d9cf-4094-a229-f338cb40d2a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b5e1c98cccc406f82d4f02c97037841'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:52:59.976913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 846/1665] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:52:59 2022] DELETE /identity/v3/users/0b5e1c98cccc406f82d4f02c97037841 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:00.354811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39416a10-6e2d-4b17-aa37-6d8b4d750f35 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:00.357310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-39416a10-6e2d-4b17-aa37-6d8b4d750f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:00.447616 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39416a10-6e2d-4b17-aa37-6d8b4d750f35 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:00.549939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:00.674819 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2344cc9-34c8-43a5-b030-ef87e6bf56c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:00.679154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2344cc9-34c8-43a5-b030-ef87e6bf56c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:00.680015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2344cc9-34c8-43a5-b030-ef87e6bf56c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:00.680893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2344cc9-34c8-43a5-b030-ef87e6bf56c7 admin admin] PATH_INFO: `/v3/projects/525d62607de94c819c18ef993aa97a80` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:00.683937 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2344cc9-34c8-43a5-b030-ef87e6bf56c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=525d62607de94c819c18ef993aa97a80)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:00.687016 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2344cc9-34c8-43a5-b030-ef87e6bf56c7 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:00.815212 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e2344cc9-34c8-43a5-b030-ef87e6bf56c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '525d62607de94c819c18ef993aa97a80', 'request_id': 'req-e2344cc9-34c8-43a5-b030-ef87e6bf56c7'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:00.838533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e2344cc9-34c8-43a5-b030-ef87e6bf56c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '525d62607de94c819c18ef993aa97a80', 'request_id': 'req-e2344cc9-34c8-43a5-b030-ef87e6bf56c7'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:00.846081 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e2344cc9-34c8-43a5-b030-ef87e6bf56c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '525d62607de94c819c18ef993aa97a80', 'request_id': 'req-e2344cc9-34c8-43a5-b030-ef87e6bf56c7'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:00.879552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e2344cc9-34c8-43a5-b030-ef87e6bf56c7 admin admin] The token cache is being invalidate because project 525d62607de94c819c18ef993aa97a80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:00.879552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e2344cc9-34c8-43a5-b030-ef87e6bf56c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:00.880198 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e2344cc9-34c8-43a5-b030-ef87e6bf56c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:00.884958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 820/1666] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:00 2022] DELETE /identity/v3/projects/525d62607de94c819c18ef993aa97a80 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:00.886758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945ee62-7551-4bd5-9b66-50cb11af8bc5 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:00.887123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945ee62-7551-4bd5-9b66-50cb11af8bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:00.887317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945ee62-7551-4bd5-9b66-50cb11af8bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:00.946290 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-e945ee62-7551-4bd5-9b66-50cb11af8bc5 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:00.947626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e945ee62-7551-4bd5-9b66-50cb11af8bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:01.087442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 847/1667] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:01.096680 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512f192-fd78-44df-8253-bb200fedfc45 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:01.097202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512f192-fd78-44df-8253-bb200fedfc45 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:01.097665 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512f192-fd78-44df-8253-bb200fedfc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:01.149194 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-1512f192-fd78-44df-8253-bb200fedfc45 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:01.156147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1512f192-fd78-44df-8253-bb200fedfc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:01.177467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 821/1668] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:01.181783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945ee62-7551-4bd5-9b66-50cb11af8bc5 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:01.195030 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945ee62-7551-4bd5-9b66-50cb11af8bc5 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:01.208836 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4da6b9-3285-493b-a6df-94868b872d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:01.209489 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4da6b9-3285-493b-a6df-94868b872d29 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:01.209773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4da6b9-3285-493b-a6df-94868b872d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:01.210047 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4da6b9-3285-493b-a6df-94868b872d29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:01.210886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4da6b9-3285-493b-a6df-94868b872d29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:53:01.211199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4da6b9-3285-493b-a6df-94868b872d29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:01.212979 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4da6b9-3285-493b-a6df-94868b872d29 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:01.276832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512f192-fd78-44df-8253-bb200fedfc45 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:01.278474 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1512f192-fd78-44df-8253-bb200fedfc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:01.290300 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 848/1669] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:53:01.297000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4da6b9-3285-493b-a6df-94868b872d29 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:01.322919 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512f192-fd78-44df-8253-bb200fedfc45 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:01.347194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4da6b9-3285-493b-a6df-94868b872d29 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:01.348907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc6825-b6a5-4e0c-9817-6bae48488c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:01.349588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc6825-b6a5-4e0c-9817-6bae48488c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:01.349868 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc6825-b6a5-4e0c-9817-6bae48488c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:01.350147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc6825-b6a5-4e0c-9817-6bae48488c80 admin admin] PATH_INFO: `/v3/projects/44f7ba2211324ca780bf2eb2c440e925` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:01.355924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc6825-b6a5-4e0c-9817-6bae48488c80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44f7ba2211324ca780bf2eb2c440e925)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:01.368572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc6825-b6a5-4e0c-9817-6bae48488c80 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:01.372036 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d2af9-acdc-4638-9b1b-f34f43e2eb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:01.372695 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d2af9-acdc-4638-9b1b-f34f43e2eb27 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:01.372980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d2af9-acdc-4638-9b1b-f34f43e2eb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:01.373264 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d2af9-acdc-4638-9b1b-f34f43e2eb27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:01.373949 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d2af9-acdc-4638-9b1b-f34f43e2eb27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:01.375918 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d2af9-acdc-4638-9b1b-f34f43e2eb27 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:01.409648 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:01.470365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 849/1670] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:01 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:01.473660 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d2af9-acdc-4638-9b1b-f34f43e2eb27 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:01.571299 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-5cfc6825-b6a5-4e0c-9817-6bae48488c80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f7ba2211324ca780bf2eb2c440e925', 'request_id': 'req-5cfc6825-b6a5-4e0c-9817-6bae48488c80'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:01.577460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-5cfc6825-b6a5-4e0c-9817-6bae48488c80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f7ba2211324ca780bf2eb2c440e925', 'request_id': 'req-5cfc6825-b6a5-4e0c-9817-6bae48488c80'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:01.589103 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-5cfc6825-b6a5-4e0c-9817-6bae48488c80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f7ba2211324ca780bf2eb2c440e925', 'request_id': 'req-5cfc6825-b6a5-4e0c-9817-6bae48488c80'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:01.607008 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-5cfc6825-b6a5-4e0c-9817-6bae48488c80 admin admin] The token cache is being invalidate because project 44f7ba2211324ca780bf2eb2c440e925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:01.607008 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-5cfc6825-b6a5-4e0c-9817-6bae48488c80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:01.607941 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-5cfc6825-b6a5-4e0c-9817-6bae48488c80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:01.610061 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 822/1671] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:01 2022] DELETE /identity/v3/projects/44f7ba2211324ca780bf2eb2c440e925 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:01.617173 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa863d4-46d5-438a-9efc-875458879565 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:01.618670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa863d4-46d5-438a-9efc-875458879565 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:01.622595 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa863d4-46d5-438a-9efc-875458879565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:01.687196 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-ffa863d4-46d5-438a-9efc-875458879565 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:01.689401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ffa863d4-46d5-438a-9efc-875458879565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:01.734751 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d2af9-acdc-4638-9b1b-f34f43e2eb27 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:01.817988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:01.929209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 823/1672] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:01.934938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ce1ad-7b53-41c1-9c55-4b7b609b830e None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:01.935385 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ce1ad-7b53-41c1-9c55-4b7b609b830e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:01.935753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ce1ad-7b53-41c1-9c55-4b7b609b830e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:01.986337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-cc3ce1ad-7b53-41c1-9c55-4b7b609b830e None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:01.987842 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-cc3ce1ad-7b53-41c1-9c55-4b7b609b830e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:01.992525 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2639752-c935-41c2-99c8-f284f67f2c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:01.993167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2639752-c935-41c2-99c8-f284f67f2c79 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:01.993448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2639752-c935-41c2-99c8-f284f67f2c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:01.993719 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2639752-c935-41c2-99c8-f284f67f2c79 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:01.994393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2639752-c935-41c2-99c8-f284f67f2c79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:01.996527 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2639752-c935-41c2-99c8-f284f67f2c79 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:02.002146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 824/1673] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:02.003675 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ce1ad-7b53-41c1-9c55-4b7b609b830e None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:02.015756 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ce1ad-7b53-41c1-9c55-4b7b609b830e None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:02.028700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61af375-a803-4e3b-a2be-1f7030b10d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:02.029408 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61af375-a803-4e3b-a2be-1f7030b10d09 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:02.029690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61af375-a803-4e3b-a2be-1f7030b10d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:02.029968 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61af375-a803-4e3b-a2be-1f7030b10d09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:02.032910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61af375-a803-4e3b-a2be-1f7030b10d09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:53:02.033641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61af375-a803-4e3b-a2be-1f7030b10d09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:02.038884 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61af375-a803-4e3b-a2be-1f7030b10d09 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:02.061538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 850/1674] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:01 2022] POST /identity/v3/users => generated 294 bytes in 590 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:02.066315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2639752-c935-41c2-99c8-f284f67f2c79 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:02.084010 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b2639752-c935-41c2-99c8-f284f67f2c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:02.094798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 825/1675] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:53:02.095884 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61af375-a803-4e3b-a2be-1f7030b10d09 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:02.114642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2639752-c935-41c2-99c8-f284f67f2c79 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:02.122246 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61af375-a803-4e3b-a2be-1f7030b10d09 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:02.129052 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3ebeb-8986-4f8f-8ad3-0a5c9b28f687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:02.130092 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3ebeb-8986-4f8f-8ad3-0a5c9b28f687 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:02.130378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3ebeb-8986-4f8f-8ad3-0a5c9b28f687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:02.131300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3ebeb-8986-4f8f-8ad3-0a5c9b28f687 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:02.131510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3ebeb-8986-4f8f-8ad3-0a5c9b28f687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:02.134150 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3ebeb-8986-4f8f-8ad3-0a5c9b28f687 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:02.149255 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5163467-8c17-4ffa-b25e-e023c104411b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:02.149992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5163467-8c17-4ffa-b25e-e023c104411b admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:02.150278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5163467-8c17-4ffa-b25e-e023c104411b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:02.150572 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5163467-8c17-4ffa-b25e-e023c104411b admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:02.151287 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5163467-8c17-4ffa-b25e-e023c104411b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:02.153073 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5163467-8c17-4ffa-b25e-e023c104411b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:02.178589 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 851/1676] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:02.179472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5163467-8c17-4ffa-b25e-e023c104411b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:02.193009 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d5163467-8c17-4ffa-b25e-e023c104411b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:02.233993 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5163467-8c17-4ffa-b25e-e023c104411b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:02.241766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3bf63-0bce-4f86-81a6-dd67670dd13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:02.242419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3bf63-0bce-4f86-81a6-dd67670dd13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:02.242744 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3bf63-0bce-4f86-81a6-dd67670dd13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:02.243012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d3bf63-0bce-4f86-81a6-dd67670dd13b admin admin] PATH_INFO: `/v3/projects/274c9d886a614ac9928f2b43c8d83d62` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:02.244233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3bf63-0bce-4f86-81a6-dd67670dd13b admin admin] RBAC: Authorizing `identity:delete_project(project_id=274c9d886a614ac9928f2b43c8d83d62)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:02.246283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d3bf63-0bce-4f86-81a6-dd67670dd13b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:02.253855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 826/1677] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:02 2022] POST /identity/v3/projects => generated 403 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:02.258149 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3ebeb-8986-4f8f-8ad3-0a5c9b28f687 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:02.332786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-19d3bf63-0bce-4f86-81a6-dd67670dd13b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '274c9d886a614ac9928f2b43c8d83d62', 'request_id': 'req-19d3bf63-0bce-4f86-81a6-dd67670dd13b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:02.365574 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-19d3bf63-0bce-4f86-81a6-dd67670dd13b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '274c9d886a614ac9928f2b43c8d83d62', 'request_id': 'req-19d3bf63-0bce-4f86-81a6-dd67670dd13b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:02.375416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-19d3bf63-0bce-4f86-81a6-dd67670dd13b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '274c9d886a614ac9928f2b43c8d83d62', 'request_id': 'req-19d3bf63-0bce-4f86-81a6-dd67670dd13b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:02.391887 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:02.401106 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-19d3bf63-0bce-4f86-81a6-dd67670dd13b admin admin] The token cache is being invalidate because project 274c9d886a614ac9928f2b43c8d83d62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:02.401594 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-19d3bf63-0bce-4f86-81a6-dd67670dd13b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:02.402274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-19d3bf63-0bce-4f86-81a6-dd67670dd13b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:02.413922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 852/1678] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:02 2022] DELETE /identity/v3/projects/274c9d886a614ac9928f2b43c8d83d62 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:02.431730 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3bf63-0bce-4f86-81a6-dd67670dd13b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:02.437692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-19d3bf63-0bce-4f86-81a6-dd67670dd13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:02.500120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3ebeb-8986-4f8f-8ad3-0a5c9b28f687 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:02.578024 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d3bf63-0bce-4f86-81a6-dd67670dd13b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:02.589695 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c7a77-5e9e-4e63-80e2-bc335a4dda2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:02.592088 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c7a77-5e9e-4e63-80e2-bc335a4dda2a admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:02.592387 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c7a77-5e9e-4e63-80e2-bc335a4dda2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:02.592663 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c7a77-5e9e-4e63-80e2-bc335a4dda2a admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:02.593353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c7a77-5e9e-4e63-80e2-bc335a4dda2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:02.596570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409c7a77-5e9e-4e63-80e2-bc335a4dda2a admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:02.665426 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6d738-edda-4116-84d0-d9229b1d793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:02.666111 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 853/1679] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:02 2022] POST /identity/v3/users => generated 302 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:02.667916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6d738-edda-4116-84d0-d9229b1d793a admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:02.668205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6d738-edda-4116-84d0-d9229b1d793a admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:02.668482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6d738-edda-4116-84d0-d9229b1d793a admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:02.669205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6d738-edda-4116-84d0-d9229b1d793a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:02.670309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edced05e-adad-4672-8d93-b7e81150c3ce None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:02.670696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edced05e-adad-4672-8d93-b7e81150c3ce None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:02.671032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edced05e-adad-4672-8d93-b7e81150c3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:02.675798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6d738-edda-4116-84d0-d9229b1d793a admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:02.714247 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 827/1680] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 458 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:02.719133 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6d738-edda-4116-84d0-d9229b1d793a admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:02.727571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-edced05e-adad-4672-8d93-b7e81150c3ce None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:02.731497 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-edced05e-adad-4672-8d93-b7e81150c3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:02.742910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6d738-edda-4116-84d0-d9229b1d793a admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:02.760317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 854/1681] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:02.763224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edced05e-adad-4672-8d93-b7e81150c3ce None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:02.764089 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b8526-99c6-4861-a534-b6863f1c50f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:02.764753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b8526-99c6-4861-a534-b6863f1c50f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:02.766222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b8526-99c6-4861-a534-b6863f1c50f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:02.766463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b8526-99c6-4861-a534-b6863f1c50f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:02.767196 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b8526-99c6-4861-a534-b6863f1c50f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:02.769144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-edced05e-adad-4672-8d93-b7e81150c3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:02.773406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b8526-99c6-4861-a534-b6863f1c50f4 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:02.792368 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edced05e-adad-4672-8d93-b7e81150c3ce None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:02.801562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 828/1682] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:02.802927 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fadd95-0200-4812-8954-6d9996250001 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:02.803215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fadd95-0200-4812-8954-6d9996250001 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:02.803493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fadd95-0200-4812-8954-6d9996250001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:02.809690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b835b-162a-4896-b12d-625506c974d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:02.810619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b835b-162a-4896-b12d-625506c974d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:02.810733 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b835b-162a-4896-b12d-625506c974d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:02.810959 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b835b-162a-4896-b12d-625506c974d1 admin admin] PATH_INFO: `/v3/projects/a0f2fecd48bb462db20b03b246b337a7/users/10e872186e18448891fa40b59eddc69e/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:02.844989 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b835b-162a-4896-b12d-625506c974d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f2fecd48bb462db20b03b246b337a7, user_id=10e872186e18448891fa40b59eddc69e, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:02.849846 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b835b-162a-4896-b12d-625506c974d1 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:02.851533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-14fadd95-0200-4812-8954-6d9996250001 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:02.852758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-14fadd95-0200-4812-8954-6d9996250001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:02.883742 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 829/1683] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:02.885664 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fadd95-0200-4812-8954-6d9996250001 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:02.897878 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 855/1684] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:02 2022] PUT /identity/v3/projects/a0f2fecd48bb462db20b03b246b337a7/users/10e872186e18448891fa40b59eddc69e/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:02.899646 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fadd95-0200-4812-8954-6d9996250001 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:02.907305 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b835b-162a-4896-b12d-625506c974d1 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:02.989011 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b835b-162a-4896-b12d-625506c974d1 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:02.997065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e135bd-0b38-480d-bdc8-fe64c4cbe1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:02.997859 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e135bd-0b38-480d-bdc8-fe64c4cbe1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:02.997981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e135bd-0b38-480d-bdc8-fe64c4cbe1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:02.998242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e135bd-0b38-480d-bdc8-fe64c4cbe1e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:02.998990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e135bd-0b38-480d-bdc8-fe64c4cbe1e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:53:02.999297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e135bd-0b38-480d-bdc8-fe64c4cbe1e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:03.001412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e135bd-0b38-480d-bdc8-fe64c4cbe1e9 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:03.009156 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:03.016052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 856/1685] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:53:03.021732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c5986-940d-4a00-b449-17246240a771 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:03.022031 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c5986-940d-4a00-b449-17246240a771 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:03.022308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c5986-940d-4a00-b449-17246240a771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:03.056882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-539c5986-940d-4a00-b449-17246240a771 None None] MFA Rules not processed for user `10e872186e18448891fa40b59eddc69e`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:03.058171 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-539c5986-940d-4a00-b449-17246240a771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:03.109437 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:03.149488 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c2f38c-e41e-4cb7-848d-774914380f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:03.150149 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2f38c-e41e-4cb7-848d-774914380f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:03.150430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2f38c-e41e-4cb7-848d-774914380f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:03.150731 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2f38c-e41e-4cb7-848d-774914380f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:03.151444 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c2f38c-e41e-4cb7-848d-774914380f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:03.153067 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c2f38c-e41e-4cb7-848d-774914380f27 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:03.173961 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 830/1686] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:02 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:03.173961 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c2f38c-e41e-4cb7-848d-774914380f27 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:03.199207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c2f38c-e41e-4cb7-848d-774914380f27 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:03.207353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 857/1687] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:03.208516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c5986-940d-4a00-b449-17246240a771 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:03.219234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd1888-7f6d-418e-be82-b87a95827f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:03.219895 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd1888-7f6d-418e-be82-b87a95827f40 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:03.220226 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd1888-7f6d-418e-be82-b87a95827f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:03.220438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd1888-7f6d-418e-be82-b87a95827f40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:03.221625 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd1888-7f6d-418e-be82-b87a95827f40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:03.223468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd1888-7f6d-418e-be82-b87a95827f40 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:03.234677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c5986-940d-4a00-b449-17246240a771 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:03.246170 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9db9b3-c24e-47fd-8134-0e4069e71e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:03.247185 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9db9b3-c24e-47fd-8134-0e4069e71e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:03.247640 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9db9b3-c24e-47fd-8134-0e4069e71e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:03.248058 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9db9b3-c24e-47fd-8134-0e4069e71e63 admin admin] PATH_INFO: `/v3/projects/a8b7dd5d565545d5852cfdd23fea372c/users/199470371a32469baee554aeb9f9b85b/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:03.280192 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9db9b3-c24e-47fd-8134-0e4069e71e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8b7dd5d565545d5852cfdd23fea372c, user_id=199470371a32469baee554aeb9f9b85b, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:03.286850 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9db9b3-c24e-47fd-8134-0e4069e71e63 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:03.291016 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 831/1688] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:03 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:03.291016 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd1888-7f6d-418e-be82-b87a95827f40 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:03.326637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 858/1689] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:03 2022] PUT /identity/v3/projects/a8b7dd5d565545d5852cfdd23fea372c/users/199470371a32469baee554aeb9f9b85b/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:03.327832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9db9b3-c24e-47fd-8134-0e4069e71e63 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:03.406594 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9db9b3-c24e-47fd-8134-0e4069e71e63 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:03.413398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d32be-c237-4b08-86cc-40da36d6820e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:03.414716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d32be-c237-4b08-86cc-40da36d6820e admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:03.415182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d32be-c237-4b08-86cc-40da36d6820e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:03.415754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d32be-c237-4b08-86cc-40da36d6820e admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:03.416605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d32be-c237-4b08-86cc-40da36d6820e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:03.420165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d32be-c237-4b08-86cc-40da36d6820e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:03.452278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 859/1690] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:03 2022] POST /identity/v3/users => generated 283 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:03.457603 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d13d1-af01-49b3-9e2b-b601dbef611a None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:03.458073 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d13d1-af01-49b3-9e2b-b601dbef611a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:03.458642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1d13d1-af01-49b3-9e2b-b601dbef611a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:03.471185 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd1888-7f6d-418e-be82-b87a95827f40 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:03.479179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0452ae3-4792-46a1-8c20-96c80ceedf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:03.480715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0452ae3-4792-46a1-8c20-96c80ceedf77 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:03.481206 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0452ae3-4792-46a1-8c20-96c80ceedf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:03.481679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0452ae3-4792-46a1-8c20-96c80ceedf77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:03.482656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0452ae3-4792-46a1-8c20-96c80ceedf77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:03.484778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0452ae3-4792-46a1-8c20-96c80ceedf77 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:03.493369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-8b1d13d1-af01-49b3-9e2b-b601dbef611a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:03.494629 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8b1d13d1-af01-49b3-9e2b-b601dbef611a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:03.517175 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 860/1691] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:03.518131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d13d1-af01-49b3-9e2b-b601dbef611a None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:03.535189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 832/1692] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:03 2022] POST /identity/v3/projects => generated 387 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:03.536984 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0452ae3-4792-46a1-8c20-96c80ceedf77 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:03.563063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1d13d1-af01-49b3-9e2b-b601dbef611a None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:03.738678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0452ae3-4792-46a1-8c20-96c80ceedf77 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:03.754274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62929ca5-03a3-4e72-adb2-0e4f58f7e918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:03.756302 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62929ca5-03a3-4e72-adb2-0e4f58f7e918 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:03.756955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62929ca5-03a3-4e72-adb2-0e4f58f7e918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:03.758874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62929ca5-03a3-4e72-adb2-0e4f58f7e918 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:03.760432 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62929ca5-03a3-4e72-adb2-0e4f58f7e918 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:03.766453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62929ca5-03a3-4e72-adb2-0e4f58f7e918 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:03.771868 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dff9ee-51e2-476b-a799-a9b2487e3478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:03.773641 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dff9ee-51e2-476b-a799-a9b2487e3478 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:03.774753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dff9ee-51e2-476b-a799-a9b2487e3478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:03.775776 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dff9ee-51e2-476b-a799-a9b2487e3478 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:03.779107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dff9ee-51e2-476b-a799-a9b2487e3478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:03.783051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dff9ee-51e2-476b-a799-a9b2487e3478 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:03.842860 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 861/1693] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:03.843872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dff9ee-51e2-476b-a799-a9b2487e3478 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:03.846313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 833/1694] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:03 2022] POST /identity/v3/users => generated 300 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:03.851269 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62929ca5-03a3-4e72-adb2-0e4f58f7e918 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:03.851821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-44dff9ee-51e2-476b-a799-a9b2487e3478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:03.894608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62929ca5-03a3-4e72-adb2-0e4f58f7e918 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:03.912666 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad8810-4335-438b-8e7e-7190be19ae6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:03.913321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad8810-4335-438b-8e7e-7190be19ae6c admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:03.913669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad8810-4335-438b-8e7e-7190be19ae6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:03.913969 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad8810-4335-438b-8e7e-7190be19ae6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:03.915353 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad8810-4335-438b-8e7e-7190be19ae6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:03.917126 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad8810-4335-438b-8e7e-7190be19ae6c admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:03.954525 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:03.955925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 834/1695] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:03.959389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad8810-4335-438b-8e7e-7190be19ae6c admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:03.975519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:03.994084 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad8810-4335-438b-8e7e-7190be19ae6c admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:04.008751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde235f-8213-4c8e-9e6a-cb7716cba16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:04.010463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde235f-8213-4c8e-9e6a-cb7716cba16b admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:04.010774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde235f-8213-4c8e-9e6a-cb7716cba16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:04.011949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde235f-8213-4c8e-9e6a-cb7716cba16b admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:04.012684 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde235f-8213-4c8e-9e6a-cb7716cba16b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:04.015917 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dff9ee-51e2-476b-a799-a9b2487e3478 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:04.017430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde235f-8213-4c8e-9e6a-cb7716cba16b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:04.022522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ccb3d-08ea-4dab-9a28-ea03f16da737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:04.023156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ccb3d-08ea-4dab-9a28-ea03f16da737 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:04.025392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ccb3d-08ea-4dab-9a28-ea03f16da737 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:04.025695 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ccb3d-08ea-4dab-9a28-ea03f16da737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:04.051898 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ccb3d-08ea-4dab-9a28-ea03f16da737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:04.057780 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ccb3d-08ea-4dab-9a28-ea03f16da737 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:04.059041 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 835/1696] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:04.061302 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde235f-8213-4c8e-9e6a-cb7716cba16b admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:04.081817 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 862/1697] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:04.086551 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ccb3d-08ea-4dab-9a28-ea03f16da737 service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:04.094125 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde235f-8213-4c8e-9e6a-cb7716cba16b admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:04.110029 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd0839-4ec8-4544-9104-5f1ef05b1541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:04.110790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd0839-4ec8-4544-9104-5f1ef05b1541 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:04.111078 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd0839-4ec8-4544-9104-5f1ef05b1541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:04.111357 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd0839-4ec8-4544-9104-5f1ef05b1541 admin admin] PATH_INFO: `/v3/projects/d09405f2d6694da599d2b4173440e1a6/users/44e75314a29f43c5936efe6c4c887989/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:04.112071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ccb3d-08ea-4dab-9a28-ea03f16da737 service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:04.127859 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5bade-cbc8-4311-8b64-a981afbce86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:04.128883 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5bade-cbc8-4311-8b64-a981afbce86c admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:04.130993 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5bade-cbc8-4311-8b64-a981afbce86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:04.131282 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5bade-cbc8-4311-8b64-a981afbce86c admin admin] PATH_INFO: `/v3/projects/316c1b0881af4f9a9a6c5e202522a948/users/73b61a96328846258cee53d502e1bbd6/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:04.131726 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd0839-4ec8-4544-9104-5f1ef05b1541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09405f2d6694da599d2b4173440e1a6, user_id=44e75314a29f43c5936efe6c4c887989, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:04.135456 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd0839-4ec8-4544-9104-5f1ef05b1541 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:04.178048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c5bade-cbc8-4311-8b64-a981afbce86c admin admin] RBAC: Authorizing `identity:create_grant(project_id=316c1b0881af4f9a9a6c5e202522a948, user_id=73b61a96328846258cee53d502e1bbd6, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:04.181097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 836/1698] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:04 2022] PUT /identity/v3/projects/d09405f2d6694da599d2b4173440e1a6/users/44e75314a29f43c5936efe6c4c887989/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:04.183589 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c5bade-cbc8-4311-8b64-a981afbce86c admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:04.193970 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cc9f1-2740-4b44-b397-bc7cbeaeb438 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:04.194671 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cc9f1-2740-4b44-b397-bc7cbeaeb438 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:04.195511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390cc9f1-2740-4b44-b397-bc7cbeaeb438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:04.231506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 863/1699] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:04 2022] PUT /identity/v3/projects/316c1b0881af4f9a9a6c5e202522a948/users/73b61a96328846258cee53d502e1bbd6/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:04.235647 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5bade-cbc8-4311-8b64-a981afbce86c admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:04.260480 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-390cc9f1-2740-4b44-b397-bc7cbeaeb438 None None] MFA Rules not processed for user `44e75314a29f43c5936efe6c4c887989`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:04.261741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-390cc9f1-2740-4b44-b397-bc7cbeaeb438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:04.372312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5bade-cbc8-4311-8b64-a981afbce86c admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:04.390444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ead1db-d48d-4296-bd4c-b820595dd009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:04.391115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ead1db-d48d-4296-bd4c-b820595dd009 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:04.391397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ead1db-d48d-4296-bd4c-b820595dd009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:04.391664 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ead1db-d48d-4296-bd4c-b820595dd009 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:04.393003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ead1db-d48d-4296-bd4c-b820595dd009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:04.394753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ead1db-d48d-4296-bd4c-b820595dd009 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:04.413526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 864/1700] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:04.418971 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ead1db-d48d-4296-bd4c-b820595dd009 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:04.436612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ead1db-d48d-4296-bd4c-b820595dd009 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:04.448303 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22fbe25-2c94-4df6-87e4-f68c6f0f7d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:04.448891 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fbe25-2c94-4df6-87e4-f68c6f0f7d4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:04.450349 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fbe25-2c94-4df6-87e4-f68c6f0f7d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:04.451419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22fbe25-2c94-4df6-87e4-f68c6f0f7d4d admin admin] PATH_INFO: `/v3/projects/316c1b0881af4f9a9a6c5e202522a948/users/73b61a96328846258cee53d502e1bbd6/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:04.455204 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fbe25-2c94-4df6-87e4-f68c6f0f7d4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=316c1b0881af4f9a9a6c5e202522a948, user_id=73b61a96328846258cee53d502e1bbd6, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:04.457011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 837/1701] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:53:04.461414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cc9f1-2740-4b44-b397-bc7cbeaeb438 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:04.464396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22fbe25-2c94-4df6-87e4-f68c6f0f7d4d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:04.484624 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390cc9f1-2740-4b44-b397-bc7cbeaeb438 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:04.490813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 865/1702] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:04 2022] PUT /identity/v3/projects/316c1b0881af4f9a9a6c5e202522a948/users/73b61a96328846258cee53d502e1bbd6/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:04.500868 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ec438-2bfb-4264-a8fe-3f30f8215ca5 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:04.501168 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ec438-2bfb-4264-a8fe-3f30f8215ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:04.501422 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ec438-2bfb-4264-a8fe-3f30f8215ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:04.557121 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-7d6ec438-2bfb-4264-a8fe-3f30f8215ca5 None None] MFA Rules not processed for user `73b61a96328846258cee53d502e1bbd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:04.558433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-7d6ec438-2bfb-4264-a8fe-3f30f8215ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:04.579002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e4565-9da9-4979-bb21-c3af7e46aaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:04.579699 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e4565-9da9-4979-bb21-c3af7e46aaa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:04.579982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e4565-9da9-4979-bb21-c3af7e46aaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:04.580255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e4565-9da9-4979-bb21-c3af7e46aaa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:04.580999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e4565-9da9-4979-bb21-c3af7e46aaa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:04.583565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e4565-9da9-4979-bb21-c3af7e46aaa6 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:04.630395 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 838/1703] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:04 2022] POST /identity/v3/projects => generated 365 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:04.634156 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e4565-9da9-4979-bb21-c3af7e46aaa6 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:04.742154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:04.771455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e4565-9da9-4979-bb21-c3af7e46aaa6 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:04.773631 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 866/1704] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:04.784234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ec438-2bfb-4264-a8fe-3f30f8215ca5 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:04.786679 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-7d6ec438-2bfb-4264-a8fe-3f30f8215ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:04.802653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76404e5e-ed40-45e4-9510-3b36affc4ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:04.803898 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76404e5e-ed40-45e4-9510-3b36affc4ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:04.804698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76404e5e-ed40-45e4-9510-3b36affc4ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:04.804976 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76404e5e-ed40-45e4-9510-3b36affc4ba5 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:04.808657 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76404e5e-ed40-45e4-9510-3b36affc4ba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:04.824134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76404e5e-ed40-45e4-9510-3b36affc4ba5 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:04.848635 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:04.891683 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 839/1705] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:04 2022] POST /identity/v3/users => generated 289 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:04.899607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76404e5e-ed40-45e4-9510-3b36affc4ba5 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:04.935015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76404e5e-ed40-45e4-9510-3b36affc4ba5 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:04.949896 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34033ff1-45b6-40f3-a449-4dc986af92e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:04.950597 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34033ff1-45b6-40f3-a449-4dc986af92e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:04.950879 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34033ff1-45b6-40f3-a449-4dc986af92e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:04.951229 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34033ff1-45b6-40f3-a449-4dc986af92e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:04.963138 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34033ff1-45b6-40f3-a449-4dc986af92e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:04.971184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34033ff1-45b6-40f3-a449-4dc986af92e1 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:04.988646 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ec438-2bfb-4264-a8fe-3f30f8215ca5 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:05.018422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 840/1706] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:05.021744 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34033ff1-45b6-40f3-a449-4dc986af92e1 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:05.058900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34033ff1-45b6-40f3-a449-4dc986af92e1 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:05.083235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94483214-76db-4a4c-97fc-f694f844513d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:05.083933 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94483214-76db-4a4c-97fc-f694f844513d admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:05.084210 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94483214-76db-4a4c-97fc-f694f844513d admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:05.084622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94483214-76db-4a4c-97fc-f694f844513d admin admin] PATH_INFO: `/v3/projects/b760a89d7ad8429e88be87f0d44562f9/users/20fdf5be1e6848498b07c943466f7183/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:05.145420 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94483214-76db-4a4c-97fc-f694f844513d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b760a89d7ad8429e88be87f0d44562f9, user_id=20fdf5be1e6848498b07c943466f7183, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:05.153097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94483214-76db-4a4c-97fc-f694f844513d admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:05.161886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7691d-bad7-496c-b696-a7bf546ca867 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:05.164264 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7691d-bad7-496c-b696-a7bf546ca867 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:05.164725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7691d-bad7-496c-b696-a7bf546ca867 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:05.165005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7691d-bad7-496c-b696-a7bf546ca867 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:05.182992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 841/1707] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:05 2022] PUT /identity/v3/projects/b760a89d7ad8429e88be87f0d44562f9/users/20fdf5be1e6848498b07c943466f7183/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:05.187128 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94483214-76db-4a4c-97fc-f694f844513d admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:05.299087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94483214-76db-4a4c-97fc-f694f844513d admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:05.307877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cadea-39d6-4639-9136-962b8edc927f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:05.308594 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cadea-39d6-4639-9136-962b8edc927f admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:05.309007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cadea-39d6-4639-9136-962b8edc927f admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:05.309287 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cadea-39d6-4639-9136-962b8edc927f admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:05.310358 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cadea-39d6-4639-9136-962b8edc927f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:05.312600 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cadea-39d6-4639-9136-962b8edc927f admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:05.327413 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 842/1708] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:05 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:05.329857 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d7691d-bad7-496c-b696-a7bf546ca867 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:05.332452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cadea-39d6-4639-9136-962b8edc927f admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:05.333644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d7691d-bad7-496c-b696-a7bf546ca867 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:05.355616 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cadea-39d6-4639-9136-962b8edc927f admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:05.363996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92259747-e4c0-4b07-8166-a1d52117a4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:05.365193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92259747-e4c0-4b07-8166-a1d52117a4a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:05.365762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92259747-e4c0-4b07-8166-a1d52117a4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:05.366261 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92259747-e4c0-4b07-8166-a1d52117a4a5 admin admin] PATH_INFO: `/v3/projects/b760a89d7ad8429e88be87f0d44562f9/users/20fdf5be1e6848498b07c943466f7183/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:05.369404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92259747-e4c0-4b07-8166-a1d52117a4a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b760a89d7ad8429e88be87f0d44562f9, user_id=20fdf5be1e6848498b07c943466f7183, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:05.371881 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 867/1709] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:05.375151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92259747-e4c0-4b07-8166-a1d52117a4a5 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:05.392922 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 843/1710] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:05 2022] PUT /identity/v3/projects/b760a89d7ad8429e88be87f0d44562f9/users/20fdf5be1e6848498b07c943466f7183/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:05.404930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539c2bb-f048-4b85-9947-ebb76cf2cfeb None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:05.405544 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539c2bb-f048-4b85-9947-ebb76cf2cfeb None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:05.406227 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539c2bb-f048-4b85-9947-ebb76cf2cfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:05.424702 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92259747-e4c0-4b07-8166-a1d52117a4a5 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:05.490470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-5539c2bb-f048-4b85-9947-ebb76cf2cfeb None None] MFA Rules not processed for user `20fdf5be1e6848498b07c943466f7183`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:05.495090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5539c2bb-f048-4b85-9947-ebb76cf2cfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:05.606618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92259747-e4c0-4b07-8166-a1d52117a4a5 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:05.622775 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f495e-936a-4068-a746-f6c230c6ca6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:05.624506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f495e-936a-4068-a746-f6c230c6ca6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:05.624688 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f495e-936a-4068-a746-f6c230c6ca6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:05.624955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f495e-936a-4068-a746-f6c230c6ca6b admin admin] PATH_INFO: `/v3/users/10e872186e18448891fa40b59eddc69e` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:05.630853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f495e-936a-4068-a746-f6c230c6ca6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=10e872186e18448891fa40b59eddc69e)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:05.633192 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f495e-936a-4068-a746-f6c230c6ca6b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:05.683609 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 868/1711] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:53:05.692581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:05.696526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539c2bb-f048-4b85-9947-ebb76cf2cfeb None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:05.698550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5539c2bb-f048-4b85-9947-ebb76cf2cfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:05.740954 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:05.756240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ea5f495e-936a-4068-a746-f6c230c6ca6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10e872186e18448891fa40b59eddc69e'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:05.787626 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ea5f495e-936a-4068-a746-f6c230c6ca6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10e872186e18448891fa40b59eddc69e'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:05.807087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ea5f495e-936a-4068-a746-f6c230c6ca6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10e872186e18448891fa40b59eddc69e'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:05.807717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ea5f495e-936a-4068-a746-f6c230c6ca6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10e872186e18448891fa40b59eddc69e'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:05.821570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ea5f495e-936a-4068-a746-f6c230c6ca6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10e872186e18448891fa40b59eddc69e'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:05.839268 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 844/1712] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:05 2022] DELETE /identity/v3/users/10e872186e18448891fa40b59eddc69e => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:05.840774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9be9bf-72de-41bd-87d5-cdfd23e3e9b7 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:05.841062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9be9bf-72de-41bd-87d5-cdfd23e3e9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:05.841871 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9be9bf-72de-41bd-87d5-cdfd23e3e9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:05.869246 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539c2bb-f048-4b85-9947-ebb76cf2cfeb None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:05.898678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-bb9be9bf-72de-41bd-87d5-cdfd23e3e9b7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:05.899949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-bb9be9bf-72de-41bd-87d5-cdfd23e3e9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:06.001201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175b399-ecc1-49e3-808a-eb602bed4f1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:06.001877 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175b399-ecc1-49e3-808a-eb602bed4f1a service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:06.002123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175b399-ecc1-49e3-808a-eb602bed4f1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:06.002398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175b399-ecc1-49e3-808a-eb602bed4f1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:06.005667 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5175b399-ecc1-49e3-808a-eb602bed4f1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:06.062303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 845/1713] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:06.074397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9be9bf-72de-41bd-87d5-cdfd23e3e9b7 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:06.078154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-bb9be9bf-72de-41bd-87d5-cdfd23e3e9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:06.191649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175b399-ecc1-49e3-808a-eb602bed4f1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:06.193121 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175b399-ecc1-49e3-808a-eb602bed4f1a service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:06.195217 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9be9bf-72de-41bd-87d5-cdfd23e3e9b7 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:06.199695 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da606e3-f41c-4476-8848-10747194fe04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:06.201060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da606e3-f41c-4476-8848-10747194fe04 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:06.201388 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da606e3-f41c-4476-8848-10747194fe04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:06.201654 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da606e3-f41c-4476-8848-10747194fe04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:06.213345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 869/1714] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:53:06.217046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58cc45-a528-4db3-a582-8a976f79f55f None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:06.217505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58cc45-a528-4db3-a582-8a976f79f55f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:06.217925 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58cc45-a528-4db3-a582-8a976f79f55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:06.224106 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da606e3-f41c-4476-8848-10747194fe04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:06.225728 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da606e3-f41c-4476-8848-10747194fe04 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:06.240160 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 846/1715] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:06.267508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-aa58cc45-a528-4db3-a582-8a976f79f55f None None] MFA Rules not processed for user `199470371a32469baee554aeb9f9b85b`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:06.269647 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-aa58cc45-a528-4db3-a582-8a976f79f55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:06.375928 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da606e3-f41c-4476-8848-10747194fe04 service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:06.377508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1da606e3-f41c-4476-8848-10747194fe04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:06.400292 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da606e3-f41c-4476-8848-10747194fe04 service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:06.411822 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63989e2a-a21c-4ab6-aa7e-fc31ab02ea2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:06.412656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63989e2a-a21c-4ab6-aa7e-fc31ab02ea2c service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:06.413506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63989e2a-a21c-4ab6-aa7e-fc31ab02ea2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:06.414423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63989e2a-a21c-4ab6-aa7e-fc31ab02ea2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:06.419475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-63989e2a-a21c-4ab6-aa7e-fc31ab02ea2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:06.445214 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 870/1716] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:06 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:53:06.451490 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58cc45-a528-4db3-a582-8a976f79f55f None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:06.453791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-aa58cc45-a528-4db3-a582-8a976f79f55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:06.473186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:06.483241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58cc45-a528-4db3-a582-8a976f79f55f None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:06.498765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e80132-5855-497b-8935-55e9ed150918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:06.499949 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e80132-5855-497b-8935-55e9ed150918 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:06.500724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e80132-5855-497b-8935-55e9ed150918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:06.501155 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e80132-5855-497b-8935-55e9ed150918 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:06.502160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e80132-5855-497b-8935-55e9ed150918 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:06.505874 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e80132-5855-497b-8935-55e9ed150918 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:06.563367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:06.574548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 871/1717] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:06 2022] POST /identity/v3/projects => generated 401 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:06.576605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e80132-5855-497b-8935-55e9ed150918 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:06.612205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63989e2a-a21c-4ab6-aa7e-fc31ab02ea2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:06.614760 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63989e2a-a21c-4ab6-aa7e-fc31ab02ea2c service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:06.658646 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 847/1718] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:06.701877 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e80132-5855-497b-8935-55e9ed150918 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:06.709710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da2997-7ac5-422f-b383-ed3fef0c7ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:06.710450 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da2997-7ac5-422f-b383-ed3fef0c7ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:06.710860 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da2997-7ac5-422f-b383-ed3fef0c7ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:06.711856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da2997-7ac5-422f-b383-ed3fef0c7ca1 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:06.712633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da2997-7ac5-422f-b383-ed3fef0c7ca1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:06.716430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da2997-7ac5-422f-b383-ed3fef0c7ca1 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:06.798985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63989e2a-a21c-4ab6-aa7e-fc31ab02ea2c service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:06.802206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 872/1719] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:06 2022] POST /identity/v3/users => generated 307 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:06.816481 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63989e2a-a21c-4ab6-aa7e-fc31ab02ea2c service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:06.828857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1406c-45eb-4776-87a7-af65b602b141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:06.829512 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1406c-45eb-4776-87a7-af65b602b141 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:06.829793 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1406c-45eb-4776-87a7-af65b602b141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:06.830065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1406c-45eb-4776-87a7-af65b602b141 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:06.830805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1406c-45eb-4776-87a7-af65b602b141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:06.833044 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1406c-45eb-4776-87a7-af65b602b141 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:06.862833 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da2997-7ac5-422f-b383-ed3fef0c7ca1 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:06.863263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 848/1720] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:06.880616 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da2997-7ac5-422f-b383-ed3fef0c7ca1 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:06.889255 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727da658-6d2b-47c5-aa41-0327ac3aa12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:06.889960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727da658-6d2b-47c5-aa41-0327ac3aa12d admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:06.890238 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727da658-6d2b-47c5-aa41-0327ac3aa12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:06.890528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727da658-6d2b-47c5-aa41-0327ac3aa12d admin admin] PATH_INFO: `/v3/projects/d65297d6db9641f6bdfe47ef79b657d5/users/55578411e8c94a1fb4ce8c2a3222a72f/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:06.923391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727da658-6d2b-47c5-aa41-0327ac3aa12d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d65297d6db9641f6bdfe47ef79b657d5, user_id=55578411e8c94a1fb4ce8c2a3222a72f, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:06.926180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727da658-6d2b-47c5-aa41-0327ac3aa12d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:06.948205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1406c-45eb-4776-87a7-af65b602b141 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:06.948938 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 873/1721] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:06 2022] PUT /identity/v3/projects/d65297d6db9641f6bdfe47ef79b657d5/users/55578411e8c94a1fb4ce8c2a3222a72f/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:06.969939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1406c-45eb-4776-87a7-af65b602b141 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:07.094430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c1197-7521-4ef3-9c5b-6f28678aad36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:07.095518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c1197-7521-4ef3-9c5b-6f28678aad36 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:07.096105 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c1197-7521-4ef3-9c5b-6f28678aad36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:07.096560 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c1197-7521-4ef3-9c5b-6f28678aad36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:07.097983 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c1197-7521-4ef3-9c5b-6f28678aad36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:07.100047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c1197-7521-4ef3-9c5b-6f28678aad36 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:07.119550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727da658-6d2b-47c5-aa41-0327ac3aa12d admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:07.120230 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 849/1722] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:06 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:07.136239 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727da658-6d2b-47c5-aa41-0327ac3aa12d admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:07.149175 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07484488-03fd-432d-917d-a027be5830fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:07.150128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07484488-03fd-432d-917d-a027be5830fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:07.152057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07484488-03fd-432d-917d-a027be5830fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:07.152523 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07484488-03fd-432d-917d-a027be5830fb admin admin] PATH_INFO: `/v3/projects/d65297d6db9641f6bdfe47ef79b657d5/users/55578411e8c94a1fb4ce8c2a3222a72f/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:07.158728 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07484488-03fd-432d-917d-a027be5830fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d65297d6db9641f6bdfe47ef79b657d5, user_id=55578411e8c94a1fb4ce8c2a3222a72f, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:07.163136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07484488-03fd-432d-917d-a027be5830fb admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:07.180209 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 874/1723] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:07 2022] PUT /identity/v3/projects/d65297d6db9641f6bdfe47ef79b657d5/users/55578411e8c94a1fb4ce8c2a3222a72f/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:07.588764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c1197-7521-4ef3-9c5b-6f28678aad36 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:07.590635 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-661c1197-7521-4ef3-9c5b-6f28678aad36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:07.624547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c1197-7521-4ef3-9c5b-6f28678aad36 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:07.746981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90b417-ec46-4845-b373-02cb6153c104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:07.747823 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90b417-ec46-4845-b373-02cb6153c104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:07.748111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90b417-ec46-4845-b373-02cb6153c104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:07.748388 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90b417-ec46-4845-b373-02cb6153c104 admin admin] PATH_INFO: `/v3/users/4a4d8a2066094a56afbae522f0629473` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:07.750666 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90b417-ec46-4845-b373-02cb6153c104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a4d8a2066094a56afbae522f0629473)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:07.752742 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90b417-ec46-4845-b373-02cb6153c104 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:07.819670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07484488-03fd-432d-917d-a027be5830fb admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:07.823128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-07484488-03fd-432d-917d-a027be5830fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:07.833464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ee90b417-ec46-4845-b373-02cb6153c104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a4d8a2066094a56afbae522f0629473'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:07.851528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ee90b417-ec46-4845-b373-02cb6153c104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a4d8a2066094a56afbae522f0629473'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:07.855011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:07.856144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07484488-03fd-432d-917d-a027be5830fb admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:07.897565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e652d2-9362-408a-a416-d8f648f35bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:07.898386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e652d2-9362-408a-a416-d8f648f35bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:07.898697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e652d2-9362-408a-a416-d8f648f35bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:07.898975 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e652d2-9362-408a-a416-d8f648f35bf7 admin admin] PATH_INFO: `/v3/projects/a0f2fecd48bb462db20b03b246b337a7` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:07.902849 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e652d2-9362-408a-a416-d8f648f35bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f2fecd48bb462db20b03b246b337a7)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:07.905594 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e652d2-9362-408a-a416-d8f648f35bf7 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:07.927849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ee90b417-ec46-4845-b373-02cb6153c104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a4d8a2066094a56afbae522f0629473'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:07.930292 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ee90b417-ec46-4845-b373-02cb6153c104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a4d8a2066094a56afbae522f0629473'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:07.953996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ee90b417-ec46-4845-b373-02cb6153c104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a4d8a2066094a56afbae522f0629473'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:07.964803 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 850/1724] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:07 2022] DELETE /identity/v3/users/4a4d8a2066094a56afbae522f0629473 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:07.982607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f34512-e36e-4aca-8a2a-248c7e6c3867 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:07.983125 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f34512-e36e-4aca-8a2a-248c7e6c3867 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:07.983548 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f34512-e36e-4aca-8a2a-248c7e6c3867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:08.040155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-c3f34512-e36e-4aca-8a2a-248c7e6c3867 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:08.047830 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c3f34512-e36e-4aca-8a2a-248c7e6c3867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:08.138665 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-b3e652d2-9362-408a-a416-d8f648f35bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f2fecd48bb462db20b03b246b337a7', 'request_id': 'req-b3e652d2-9362-408a-a416-d8f648f35bf7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:08.164047 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-b3e652d2-9362-408a-a416-d8f648f35bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f2fecd48bb462db20b03b246b337a7', 'request_id': 'req-b3e652d2-9362-408a-a416-d8f648f35bf7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:08.187233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-b3e652d2-9362-408a-a416-d8f648f35bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f2fecd48bb462db20b03b246b337a7', 'request_id': 'req-b3e652d2-9362-408a-a416-d8f648f35bf7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:08.214706 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-b3e652d2-9362-408a-a416-d8f648f35bf7 admin admin] The token cache is being invalidate because project a0f2fecd48bb462db20b03b246b337a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:08.216032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-b3e652d2-9362-408a-a416-d8f648f35bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:08.217698 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-b3e652d2-9362-408a-a416-d8f648f35bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:08.219849 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 875/1725] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:07 2022] DELETE /identity/v3/projects/a0f2fecd48bb462db20b03b246b337a7 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:08.227797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e652d2-9362-408a-a416-d8f648f35bf7 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:08.235608 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b3e652d2-9362-408a-a416-d8f648f35bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:08.248943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 851/1726] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:53:08.288185 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:08.296837 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f34512-e36e-4aca-8a2a-248c7e6c3867 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:08.298352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c3f34512-e36e-4aca-8a2a-248c7e6c3867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:08.427630 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e652d2-9362-408a-a416-d8f648f35bf7 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:08.449564 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded84a4-ff92-421d-bb9e-6f7698503135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:08.450251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded84a4-ff92-421d-bb9e-6f7698503135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:08.450737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded84a4-ff92-421d-bb9e-6f7698503135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:08.450957 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded84a4-ff92-421d-bb9e-6f7698503135 admin admin] PATH_INFO: `/v3/users/73b61a96328846258cee53d502e1bbd6` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:08.455592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded84a4-ff92-421d-bb9e-6f7698503135 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73b61a96328846258cee53d502e1bbd6)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:08.463150 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded84a4-ff92-421d-bb9e-6f7698503135 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:08.479397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f34512-e36e-4aca-8a2a-248c7e6c3867 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:08.580970 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-dded84a4-ff92-421d-bb9e-6f7698503135 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73b61a96328846258cee53d502e1bbd6'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:08.600011 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-dded84a4-ff92-421d-bb9e-6f7698503135 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73b61a96328846258cee53d502e1bbd6'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:08.630080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-dded84a4-ff92-421d-bb9e-6f7698503135 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73b61a96328846258cee53d502e1bbd6'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:08.632221 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-dded84a4-ff92-421d-bb9e-6f7698503135 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73b61a96328846258cee53d502e1bbd6'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:08.634320 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-dded84a4-ff92-421d-bb9e-6f7698503135 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73b61a96328846258cee53d502e1bbd6'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:08.647296 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922f9cc-c31b-4a2f-8976-9265a43e112e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:08.648191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922f9cc-c31b-4a2f-8976-9265a43e112e service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:08.648191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922f9cc-c31b-4a2f-8976-9265a43e112e service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:08.648530 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922f9cc-c31b-4a2f-8976-9265a43e112e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:08.652666 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c922f9cc-c31b-4a2f-8976-9265a43e112e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:08.673936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 876/1727] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:08 2022] DELETE /identity/v3/users/73b61a96328846258cee53d502e1bbd6 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:08.689830 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922f9cc-c31b-4a2f-8976-9265a43e112e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:08.692040 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922f9cc-c31b-4a2f-8976-9265a43e112e service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:08.715344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 852/1728] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:53:08.973874 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f8411-15a5-4ee8-99f6-820ac8dde635 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:08.974416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f8411-15a5-4ee8-99f6-820ac8dde635 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:08.974416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f8411-15a5-4ee8-99f6-820ac8dde635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:09.031215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-ec7f8411-15a5-4ee8-99f6-820ac8dde635 None None] MFA Rules not processed for user `55578411e8c94a1fb4ce8c2a3222a72f`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:09.035600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ec7f8411-15a5-4ee8-99f6-820ac8dde635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:09.181719 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 877/1729] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:08 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:09.678676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922f9cc-c31b-4a2f-8976-9265a43e112e service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:09.681399 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c922f9cc-c31b-4a2f-8976-9265a43e112e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:09.696197 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922f9cc-c31b-4a2f-8976-9265a43e112e service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:09.798848 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:09.827538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4d482-1834-4b9e-a765-03816f7173a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:09.828574 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4d482-1834-4b9e-a765-03816f7173a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:09.829128 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4d482-1834-4b9e-a765-03816f7173a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:09.829812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4d482-1834-4b9e-a765-03816f7173a9 admin admin] PATH_INFO: `/v3/projects/6ca733bcc6dd48afb6e0fcb62b56b9f9` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:09.832248 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4d482-1834-4b9e-a765-03816f7173a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ca733bcc6dd48afb6e0fcb62b56b9f9)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:09.834959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4d482-1834-4b9e-a765-03816f7173a9 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:09.926344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f8411-15a5-4ee8-99f6-820ac8dde635 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:09.928180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ec7f8411-15a5-4ee8-99f6-820ac8dde635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:09.954649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f8411-15a5-4ee8-99f6-820ac8dde635 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:09.962146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-7ba4d482-1834-4b9e-a765-03816f7173a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ca733bcc6dd48afb6e0fcb62b56b9f9', 'request_id': 'req-7ba4d482-1834-4b9e-a765-03816f7173a9'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:09.992073 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-7ba4d482-1834-4b9e-a765-03816f7173a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ca733bcc6dd48afb6e0fcb62b56b9f9', 'request_id': 'req-7ba4d482-1834-4b9e-a765-03816f7173a9'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:10.011237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-7ba4d482-1834-4b9e-a765-03816f7173a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ca733bcc6dd48afb6e0fcb62b56b9f9', 'request_id': 'req-7ba4d482-1834-4b9e-a765-03816f7173a9'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:10.021104 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:10.041680 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-7ba4d482-1834-4b9e-a765-03816f7173a9 admin admin] The token cache is being invalidate because project 6ca733bcc6dd48afb6e0fcb62b56b9f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:10.043859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-7ba4d482-1834-4b9e-a765-03816f7173a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:10.045278 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-7ba4d482-1834-4b9e-a765-03816f7173a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:10.048264 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 853/1730] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:09 2022] DELETE /identity/v3/projects/6ca733bcc6dd48afb6e0fcb62b56b9f9 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:10.058147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa1562-ec3d-4a2d-a8b9-057377b789b5 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:10.059707 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa1562-ec3d-4a2d-a8b9-057377b789b5 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:10.060210 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa1562-ec3d-4a2d-a8b9-057377b789b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:10.119899 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-4aaa1562-ec3d-4a2d-a8b9-057377b789b5 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:10.123644 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-4aaa1562-ec3d-4a2d-a8b9-057377b789b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:10.140452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e222dc-13a4-4db8-b033-6b736dc11334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:10.141586 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e222dc-13a4-4db8-b033-6b736dc11334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:10.142156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e222dc-13a4-4db8-b033-6b736dc11334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:10.142651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e222dc-13a4-4db8-b033-6b736dc11334 admin admin] PATH_INFO: `/v3/projects/316c1b0881af4f9a9a6c5e202522a948` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:10.145818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e222dc-13a4-4db8-b033-6b736dc11334 admin admin] RBAC: Authorizing `identity:delete_project(project_id=316c1b0881af4f9a9a6c5e202522a948)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:10.149150 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e222dc-13a4-4db8-b033-6b736dc11334 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:10.258259 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 854/1731] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:10.266563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6bf22-2a47-4a3e-9683-1855e4e0eed0 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:10.267164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6bf22-2a47-4a3e-9683-1855e4e0eed0 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:10.267736 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6bf22-2a47-4a3e-9683-1855e4e0eed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:10.290263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-32e222dc-13a4-4db8-b033-6b736dc11334 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '316c1b0881af4f9a9a6c5e202522a948', 'request_id': 'req-32e222dc-13a4-4db8-b033-6b736dc11334'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:10.301820 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-32e222dc-13a4-4db8-b033-6b736dc11334 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '316c1b0881af4f9a9a6c5e202522a948', 'request_id': 'req-32e222dc-13a4-4db8-b033-6b736dc11334'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:10.313178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-32e222dc-13a4-4db8-b033-6b736dc11334 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '316c1b0881af4f9a9a6c5e202522a948', 'request_id': 'req-32e222dc-13a4-4db8-b033-6b736dc11334'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:10.330564 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-32e222dc-13a4-4db8-b033-6b736dc11334 admin admin] The token cache is being invalidate because project 316c1b0881af4f9a9a6c5e202522a948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:10.331038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-32e222dc-13a4-4db8-b033-6b736dc11334 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:10.332838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-32e222dc-13a4-4db8-b033-6b736dc11334 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:10.335222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 878/1732] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:09 2022] DELETE /identity/v3/projects/316c1b0881af4f9a9a6c5e202522a948 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:10.339757 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a4a6bf22-2a47-4a3e-9683-1855e4e0eed0 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:10.340011 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092eeca-2b3a-4836-be9e-3b1064e703d6 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:10.340192 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092eeca-2b3a-4836-be9e-3b1064e703d6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:10.340469 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092eeca-2b3a-4836-be9e-3b1064e703d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:10.343810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a4a6bf22-2a47-4a3e-9683-1855e4e0eed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:10.367819 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1092eeca-2b3a-4836-be9e-3b1064e703d6 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:10.368803 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1092eeca-2b3a-4836-be9e-3b1064e703d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:10.465489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 855/1733] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:10.469902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6bf22-2a47-4a3e-9683-1855e4e0eed0 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:10.485615 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 879/1734] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:10.492638 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6bf22-2a47-4a3e-9683-1855e4e0eed0 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:10.508543 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0d65c-a7e1-40d6-83ea-f38799c4f433 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:10.508543 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0d65c-a7e1-40d6-83ea-f38799c4f433 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:10.508543 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0d65c-a7e1-40d6-83ea-f38799c4f433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:10.518356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786cd49-1bb4-4b1d-b465-aa5197478a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:10.519208 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786cd49-1bb4-4b1d-b465-aa5197478a61 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:10.519443 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786cd49-1bb4-4b1d-b465-aa5197478a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:10.519747 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786cd49-1bb4-4b1d-b465-aa5197478a61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:10.520502 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786cd49-1bb4-4b1d-b465-aa5197478a61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:53:10.520736 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786cd49-1bb4-4b1d-b465-aa5197478a61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:10.522624 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9786cd49-1bb4-4b1d-b465-aa5197478a61 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:10.539701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-12d0d65c-a7e1-40d6-83ea-f38799c4f433 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:10.540945 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-12d0d65c-a7e1-40d6-83ea-f38799c4f433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:10.553773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 856/1735] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:53:10.555768 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786cd49-1bb4-4b1d-b465-aa5197478a61 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:10.559100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-9786cd49-1bb4-4b1d-b465-aa5197478a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:10.563383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 880/1736] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:53:10.564689 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a4839-fcea-41d2-b3e1-771975e9e352 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:10.564977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a4839-fcea-41d2-b3e1-771975e9e352 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:10.565257 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a4839-fcea-41d2-b3e1-771975e9e352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:10.569850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:10.576449 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786cd49-1bb4-4b1d-b465-aa5197478a61 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:10.584690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd3cca-b16f-4eda-ac94-f5e1f18d4721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:10.585698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd3cca-b16f-4eda-ac94-f5e1f18d4721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:10.586128 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd3cca-b16f-4eda-ac94-f5e1f18d4721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:10.587314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd3cca-b16f-4eda-ac94-f5e1f18d4721 admin admin] PATH_INFO: `/v3/users/44e75314a29f43c5936efe6c4c887989` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:10.588968 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd3cca-b16f-4eda-ac94-f5e1f18d4721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44e75314a29f43c5936efe6c4c887989)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:10.591082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd3cca-b16f-4eda-ac94-f5e1f18d4721 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:10.647583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-fd7a4839-fcea-41d2-b3e1-771975e9e352 None None] MFA Rules not processed for user `44fbd50cb5664149814abf0e9e44e18f`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:10.648836 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-fd7a4839-fcea-41d2-b3e1-771975e9e352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:10.753994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:10.760637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9cdd3cca-b16f-4eda-ac94-f5e1f18d4721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44e75314a29f43c5936efe6c4c887989'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:10.801201 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9cdd3cca-b16f-4eda-ac94-f5e1f18d4721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44e75314a29f43c5936efe6c4c887989'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:10.834348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9cdd3cca-b16f-4eda-ac94-f5e1f18d4721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44e75314a29f43c5936efe6c4c887989'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:10.835926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9cdd3cca-b16f-4eda-ac94-f5e1f18d4721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44e75314a29f43c5936efe6c4c887989'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:10.844970 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 881/1737] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:10.845996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a4839-fcea-41d2-b3e1-771975e9e352 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:10.875553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a4839-fcea-41d2-b3e1-771975e9e352 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:10.882446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9cdd3cca-b16f-4eda-ac94-f5e1f18d4721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44e75314a29f43c5936efe6c4c887989'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:10.893333 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfeeb2-2cd5-4b0c-bb14-9c6335f12ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:10.894088 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfeeb2-2cd5-4b0c-bb14-9c6335f12ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:10.895818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfeeb2-2cd5-4b0c-bb14-9c6335f12ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:10.896111 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfeeb2-2cd5-4b0c-bb14-9c6335f12ba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:10.896811 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfeeb2-2cd5-4b0c-bb14-9c6335f12ba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:10.900939 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfeeb2-2cd5-4b0c-bb14-9c6335f12ba9 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:10.917474 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 857/1738] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:10 2022] DELETE /identity/v3/users/44e75314a29f43c5936efe6c4c887989 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:10.923589 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd3cca-b16f-4eda-ac94-f5e1f18d4721 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:10.931160 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-9cdd3cca-b16f-4eda-ac94-f5e1f18d4721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:10.983312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 882/1739] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:10 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:10.992736 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e44231-95ce-418a-85bb-e6d6648631e3 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:10.994632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e44231-95ce-418a-85bb-e6d6648631e3 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:10.995054 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e44231-95ce-418a-85bb-e6d6648631e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:11.068999 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-10e44231-95ce-418a-85bb-e6d6648631e3 None None] MFA Rules not processed for user `5f16ff28aac54514a475f8a4f5c6e4c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:11.078305 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-10e44231-95ce-418a-85bb-e6d6648631e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:11.110136 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd3cca-b16f-4eda-ac94-f5e1f18d4721 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:11.249009 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21056674-ffe4-4b38-bb3b-14128d97b101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:11.249662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21056674-ffe4-4b38-bb3b-14128d97b101 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:11.250023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21056674-ffe4-4b38-bb3b-14128d97b101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:11.250195 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21056674-ffe4-4b38-bb3b-14128d97b101 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:11.250943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21056674-ffe4-4b38-bb3b-14128d97b101 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:53:11.251975 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21056674-ffe4-4b38-bb3b-14128d97b101 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:11.254139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21056674-ffe4-4b38-bb3b-14128d97b101 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:11.305364 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 858/1740] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:53:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:53:11.309241 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b35804-1041-4542-807b-1e22f377992d None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:11.311806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b35804-1041-4542-807b-1e22f377992d None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:11.312550 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b35804-1041-4542-807b-1e22f377992d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:11.327366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 883/1741] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:11.328663 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2eb4d-233c-45ee-9953-68c8a01befec None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:11.328956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2eb4d-233c-45ee-9953-68c8a01befec None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:11.329232 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2eb4d-233c-45ee-9953-68c8a01befec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:11.410632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-f3c2eb4d-233c-45ee-9953-68c8a01befec None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:11.414140 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f3c2eb4d-233c-45ee-9953-68c8a01befec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:11.422888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-32b35804-1041-4542-807b-1e22f377992d None None] MFA Rules not processed for user `44fbd50cb5664149814abf0e9e44e18f`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:11.424146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-32b35804-1041-4542-807b-1e22f377992d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:11.439076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 884/1742] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:11.440034 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2eb4d-233c-45ee-9953-68c8a01befec None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:11.445784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f3c2eb4d-233c-45ee-9953-68c8a01befec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:11.475286 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2eb4d-233c-45ee-9953-68c8a01befec None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:11.485256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c28421-00a2-4c25-ad65-a3287925e812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:11.488714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c28421-00a2-4c25-ad65-a3287925e812 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:11.489001 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c28421-00a2-4c25-ad65-a3287925e812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:11.490427 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c28421-00a2-4c25-ad65-a3287925e812 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:11.491249 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c28421-00a2-4c25-ad65-a3287925e812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:11.495134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:11.495962 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c28421-00a2-4c25-ad65-a3287925e812 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:11.573988 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 885/1743] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:11 2022] POST /identity/v3/users => generated 287 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:11.576099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06670d1e-8f11-4cd8-abd8-82b5d13f0e1c None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:11.576398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06670d1e-8f11-4cd8-abd8-82b5d13f0e1c None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:11.576676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06670d1e-8f11-4cd8-abd8-82b5d13f0e1c None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:11.581016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 886/1744] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 3 12:53:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:53:11.581826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06670d1e-8f11-4cd8-abd8-82b5d13f0e1c None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:11.618533 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06670d1e-8f11-4cd8-abd8-82b5d13f0e1c None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:11.634772 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 859/1745] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:11 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:11.635494 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917dee6-213b-4761-b79a-327a6ab63999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:11.636774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b35804-1041-4542-807b-1e22f377992d None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:11.644557 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917dee6-213b-4761-b79a-327a6ab63999 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:11.644850 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917dee6-213b-4761-b79a-327a6ab63999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:11.645143 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917dee6-213b-4761-b79a-327a6ab63999 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:11.645831 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917dee6-213b-4761-b79a-327a6ab63999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:11.646613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-32b35804-1041-4542-807b-1e22f377992d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:11.647806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917dee6-213b-4761-b79a-327a6ab63999 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:11.728958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 887/1746] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:11 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:11.730208 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917dee6-213b-4761-b79a-327a6ab63999 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:11.734179 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5917dee6-213b-4761-b79a-327a6ab63999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:11.877590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917dee6-213b-4761-b79a-327a6ab63999 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:11.882104 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d0d8c-5ff6-4c98-ad58-56493c630dec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:11.883322 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d0d8c-5ff6-4c98-ad58-56493c630dec service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:11.883617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d0d8c-5ff6-4c98-ad58-56493c630dec service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:11.883942 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d0d8c-5ff6-4c98-ad58-56493c630dec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:11.886385 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b35804-1041-4542-807b-1e22f377992d None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:11.897784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:11.992662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fdb6dc-3d24-40a1-b75c-2e988b8ce4b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:11.993806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdb6dc-3d24-40a1-b75c-2e988b8ce4b2 service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:11.994207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdb6dc-3d24-40a1-b75c-2e988b8ce4b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:11.994654 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdb6dc-3d24-40a1-b75c-2e988b8ce4b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:12.039849 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d0d8c-5ff6-4c98-ad58-56493c630dec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:12.044633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d0d8c-5ff6-4c98-ad58-56493c630dec service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:12.070473 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 888/1747] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:12.072010 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d0d8c-5ff6-4c98-ad58-56493c630dec service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:12.090107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:12.094845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fdb6dc-3d24-40a1-b75c-2e988b8ce4b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:12.096774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fdb6dc-3d24-40a1-b75c-2e988b8ce4b2 service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:12.113308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d0d8c-5ff6-4c98-ad58-56493c630dec service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:12.124431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 860/1748] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:12.126999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fdb6dc-3d24-40a1-b75c-2e988b8ce4b2 service nova] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:12.128994 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-86fdb6dc-3d24-40a1-b75c-2e988b8ce4b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:12.137896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c727ae-691b-40fe-8a80-2e5822d13431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:12.141677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c727ae-691b-40fe-8a80-2e5822d13431 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:12.141899 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c727ae-691b-40fe-8a80-2e5822d13431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:12.146088 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c727ae-691b-40fe-8a80-2e5822d13431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:12.147762 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c727ae-691b-40fe-8a80-2e5822d13431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:12.154918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fdb6dc-3d24-40a1-b75c-2e988b8ce4b2 service nova] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:12.155344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c727ae-691b-40fe-8a80-2e5822d13431 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:12.162990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524a24d-d457-4f79-8e6b-45df8f24f2de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:12.163635 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524a24d-d457-4f79-8e6b-45df8f24f2de service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:12.163915 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524a24d-d457-4f79-8e6b-45df8f24f2de service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:12.164194 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524a24d-d457-4f79-8e6b-45df8f24f2de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:12.166168 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2524a24d-d457-4f79-8e6b-45df8f24f2de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:12.211323 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 889/1749] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:12.215095 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ca43e-f702-4f32-bff1-7b2b25e3610d None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:12.219218 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ca43e-f702-4f32-bff1-7b2b25e3610d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:12.219681 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ca43e-f702-4f32-bff1-7b2b25e3610d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:12.272364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1a9ca43e-f702-4f32-bff1-7b2b25e3610d None None] MFA Rules not processed for user `5094263ac471481db48783b19192e01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:12.276098 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524a24d-d457-4f79-8e6b-45df8f24f2de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:12.276734 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1a9ca43e-f702-4f32-bff1-7b2b25e3610d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:12.277947 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524a24d-d457-4f79-8e6b-45df8f24f2de service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:12.292489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 861/1750] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:12.293810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524a24d-d457-4f79-8e6b-45df8f24f2de service nova] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:12.307568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524a24d-d457-4f79-8e6b-45df8f24f2de service nova] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:12.321182 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9a994-f912-469e-b127-9b7b54ccd233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:12.322050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9a994-f912-469e-b127-9b7b54ccd233 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:12.322472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9a994-f912-469e-b127-9b7b54ccd233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:12.322902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9a994-f912-469e-b127-9b7b54ccd233 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:12.323797 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9a994-f912-469e-b127-9b7b54ccd233 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:12.326476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee9a994-f912-469e-b127-9b7b54ccd233 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:12.364141 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 862/1751] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:12 2022] POST /identity/v3/users => generated 294 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:12.367815 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9a994-f912-469e-b127-9b7b54ccd233 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:12.393242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9a994-f912-469e-b127-9b7b54ccd233 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:12.408749 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d888703-858e-455f-a98f-9d7644ebc95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:12.411458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d888703-858e-455f-a98f-9d7644ebc95d admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:12.411894 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d888703-858e-455f-a98f-9d7644ebc95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:12.412410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d888703-858e-455f-a98f-9d7644ebc95d admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:12.415887 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d888703-858e-455f-a98f-9d7644ebc95d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:12.420059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d888703-858e-455f-a98f-9d7644ebc95d admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:12.452481 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 863/1752] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:12.453394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d888703-858e-455f-a98f-9d7644ebc95d admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:12.480707 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d888703-858e-455f-a98f-9d7644ebc95d admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:12.495973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924f232-b62d-4b3e-ad02-e5e2f21a8cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:12.496524 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924f232-b62d-4b3e-ad02-e5e2f21a8cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:12.496841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924f232-b62d-4b3e-ad02-e5e2f21a8cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:12.497071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924f232-b62d-4b3e-ad02-e5e2f21a8cd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:12.497783 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924f232-b62d-4b3e-ad02-e5e2f21a8cd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:12.514299 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0924f232-b62d-4b3e-ad02-e5e2f21a8cd9 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:12.547279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 890/1753] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 3 12:53:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:12.548866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ca43e-f702-4f32-bff1-7b2b25e3610d None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:12.558109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 864/1754] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:12.559176 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924f232-b62d-4b3e-ad02-e5e2f21a8cd9 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:12.577028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ca43e-f702-4f32-bff1-7b2b25e3610d None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:12.584704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0924f232-b62d-4b3e-ad02-e5e2f21a8cd9 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:12.600579 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9f782-5902-4c85-aaf5-7fd55410bcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:12.602380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9f782-5902-4c85-aaf5-7fd55410bcc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:12.604399 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9f782-5902-4c85-aaf5-7fd55410bcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:12.604764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9f782-5902-4c85-aaf5-7fd55410bcc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:12.605588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9f782-5902-4c85-aaf5-7fd55410bcc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:12.608721 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7c6ba-968a-4e1b-9185-8aa618c74ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:12.609438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7c6ba-968a-4e1b-9185-8aa618c74ff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:12.609867 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9f782-5902-4c85-aaf5-7fd55410bcc7 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:12.614026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7c6ba-968a-4e1b-9185-8aa618c74ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:12.614341 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7c6ba-968a-4e1b-9185-8aa618c74ff9 admin admin] PATH_INFO: `/v3/projects/24d2205a09d64fefad6e0d890544dab4/users/5d699ddb04ce4e4180388aa212d87840/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:12.636384 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 865/1755] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:12 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:12.647446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481e2b1-552e-47d6-b8e0-0cf0ab2e7203 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:12.647846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481e2b1-552e-47d6-b8e0-0cf0ab2e7203 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:12.647951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481e2b1-552e-47d6-b8e0-0cf0ab2e7203 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:12.656104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 866/1756] 10.222.0.65 () {54 vars in 961 bytes} [Wed Aug 3 12:53:12 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:53:12.656849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481e2b1-552e-47d6-b8e0-0cf0ab2e7203 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:12.662694 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7c6ba-968a-4e1b-9185-8aa618c74ff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24d2205a09d64fefad6e0d890544dab4, user_id=5d699ddb04ce4e4180388aa212d87840, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:12.673914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7c6ba-968a-4e1b-9185-8aa618c74ff9 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:12.697743 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481e2b1-552e-47d6-b8e0-0cf0ab2e7203 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:12.719561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 891/1757] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:12 2022] PUT /identity/v3/projects/24d2205a09d64fefad6e0d890544dab4/users/5d699ddb04ce4e4180388aa212d87840/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:12.722055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439c307-f912-4191-9103-4964f03e4a7d None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:12.722346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439c307-f912-4191-9103-4964f03e4a7d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:12.722648 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439c307-f912-4191-9103-4964f03e4a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:12.767895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-9439c307-f912-4191-9103-4964f03e4a7d None None] MFA Rules not processed for user `5094263ac471481db48783b19192e01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:12.771496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-9439c307-f912-4191-9103-4964f03e4a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:12.824706 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14105b-3258-4f12-b4d7-a174bb463286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:12.825782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14105b-3258-4f12-b4d7-a174bb463286 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:12.826339 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14105b-3258-4f12-b4d7-a174bb463286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:12.826850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14105b-3258-4f12-b4d7-a174bb463286 admin admin] PATH_INFO: `/v3/projects/9c9a46618ac346c9bd563dff92235fcd/users/9aee697b6b97434da6b4324dad02f5b2/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:12.893333 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14105b-3258-4f12-b4d7-a174bb463286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c9a46618ac346c9bd563dff92235fcd, user_id=9aee697b6b97434da6b4324dad02f5b2, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:12.896276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14105b-3258-4f12-b4d7-a174bb463286 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:12.938255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 867/1758] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:12 2022] PUT /identity/v3/projects/9c9a46618ac346c9bd563dff92235fcd/users/9aee697b6b97434da6b4324dad02f5b2/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:12.938255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb17ca5-337a-46ba-aae2-54d39f6d47c0 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:12.938255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb17ca5-337a-46ba-aae2-54d39f6d47c0 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:12.938255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb17ca5-337a-46ba-aae2-54d39f6d47c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:12.959711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 892/1759] 10.222.0.65 () {58 vars in 1058 bytes} [Wed Aug 3 12:53:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:12.961757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9439c307-f912-4191-9103-4964f03e4a7d None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:12.965065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-9439c307-f912-4191-9103-4964f03e4a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:12.979543 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:12.986802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9439c307-f912-4191-9103-4964f03e4a7d None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:12.989966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-3eb17ca5-337a-46ba-aae2-54d39f6d47c0 None None] MFA Rules not processed for user `5d699ddb04ce4e4180388aa212d87840`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:12.993768 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-3eb17ca5-337a-46ba-aae2-54d39f6d47c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:13.030979 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:13.141390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 868/1760] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:13.142583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7eb9f8-e680-4799-8160-404a9d1b4f6d None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:13.142759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31331d3f-5ff6-4bea-acad-ed2100ba72f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:13.143198 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7eb9f8-e680-4799-8160-404a9d1b4f6d None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:13.143198 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7eb9f8-e680-4799-8160-404a9d1b4f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:13.143483 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31331d3f-5ff6-4bea-acad-ed2100ba72f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:13.143956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31331d3f-5ff6-4bea-acad-ed2100ba72f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:13.144402 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31331d3f-5ff6-4bea-acad-ed2100ba72f2 admin admin] PATH_INFO: `/v3/users/199470371a32469baee554aeb9f9b85b` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:13.147502 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31331d3f-5ff6-4bea-acad-ed2100ba72f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=199470371a32469baee554aeb9f9b85b)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:13.149784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31331d3f-5ff6-4bea-acad-ed2100ba72f2 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:13.229265 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-6e7eb9f8-e680-4799-8160-404a9d1b4f6d None None] MFA Rules not processed for user `9aee697b6b97434da6b4324dad02f5b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:13.232511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6e7eb9f8-e680-4799-8160-404a9d1b4f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:13.247927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-31331d3f-5ff6-4bea-acad-ed2100ba72f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '199470371a32469baee554aeb9f9b85b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:13.270046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-31331d3f-5ff6-4bea-acad-ed2100ba72f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '199470371a32469baee554aeb9f9b85b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:13.292214 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-31331d3f-5ff6-4bea-acad-ed2100ba72f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '199470371a32469baee554aeb9f9b85b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:13.293043 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-31331d3f-5ff6-4bea-acad-ed2100ba72f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '199470371a32469baee554aeb9f9b85b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:13.293684 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-31331d3f-5ff6-4bea-acad-ed2100ba72f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '199470371a32469baee554aeb9f9b85b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:13.325273 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 893/1761] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:12 2022] DELETE /identity/v3/users/199470371a32469baee554aeb9f9b85b => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:13.326655 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31331d3f-5ff6-4bea-acad-ed2100ba72f2 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:13.341392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-31331d3f-5ff6-4bea-acad-ed2100ba72f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:13.428875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 869/1762] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:13.428875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7eb9f8-e680-4799-8160-404a9d1b4f6d None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:13.431827 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6e7eb9f8-e680-4799-8160-404a9d1b4f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:13.521164 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31331d3f-5ff6-4bea-acad-ed2100ba72f2 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:13.542665 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852a6c1-7f08-4446-bdc8-1cfaf799738b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5094263ac471481db48783b19192e01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:13.543319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852a6c1-7f08-4446-bdc8-1cfaf799738b service placement] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:13.543602 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852a6c1-7f08-4446-bdc8-1cfaf799738b service placement] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:13.543879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852a6c1-7f08-4446-bdc8-1cfaf799738b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:13.548202 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-9852a6c1-7f08-4446-bdc8-1cfaf799738b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:13.579405 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852a6c1-7f08-4446-bdc8-1cfaf799738b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:13.584040 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852a6c1-7f08-4446-bdc8-1cfaf799738b service placement] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:13.606991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 894/1763] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:13.607828 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852a6c1-7f08-4446-bdc8-1cfaf799738b service placement] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:13.611513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-9852a6c1-7f08-4446-bdc8-1cfaf799738b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:13.611646 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7eb9f8-e680-4799-8160-404a9d1b4f6d None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:13.651847 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:13.722496 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:13.769880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cba69-2db9-4b34-b442-f9aef9a0447e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:13.770550 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cba69-2db9-4b34-b442-f9aef9a0447e service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:13.770879 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cba69-2db9-4b34-b442-f9aef9a0447e service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:13.771124 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622cba69-2db9-4b34-b442-f9aef9a0447e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:13.776032 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-622cba69-2db9-4b34-b442-f9aef9a0447e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:13.804678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852a6c1-7f08-4446-bdc8-1cfaf799738b service placement] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:13.814621 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:13.817440 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:13.817785 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:13.818079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f admin admin] PATH_INFO: `/v3/projects/d09405f2d6694da599d2b4173440e1a6` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:13.819840 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d09405f2d6694da599d2b4173440e1a6)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:13.822196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:13.928588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cba69-2db9-4b34-b442-f9aef9a0447e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:13.931890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622cba69-2db9-4b34-b442-f9aef9a0447e service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:13.952267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd09405f2d6694da599d2b4173440e1a6', 'request_id': 'req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:13.963850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 870/1764] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:13.964944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cba69-2db9-4b34-b442-f9aef9a0447e service nova] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:13.967524 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-622cba69-2db9-4b34-b442-f9aef9a0447e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:13.974669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd09405f2d6694da599d2b4173440e1a6', 'request_id': 'req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:13.990709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd09405f2d6694da599d2b4173440e1a6', 'request_id': 'req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:14.012556 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622cba69-2db9-4b34-b442-f9aef9a0447e service nova] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:14.016622 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f admin admin] The token cache is being invalidate because project d09405f2d6694da599d2b4173440e1a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:14.017060 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:14.023795 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:14.028802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 895/1765] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:13 2022] DELETE /identity/v3/projects/d09405f2d6694da599d2b4173440e1a6 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:14.029699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:14.032158 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:14.173079 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07864c48-f7a0-4d9e-ac0f-7ec7b47dd447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:14.174139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07864c48-f7a0-4d9e-ac0f-7ec7b47dd447 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:14.176827 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07864c48-f7a0-4d9e-ac0f-7ec7b47dd447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:14.179995 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07864c48-f7a0-4d9e-ac0f-7ec7b47dd447 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:14.180874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07864c48-f7a0-4d9e-ac0f-7ec7b47dd447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:14.187278 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07864c48-f7a0-4d9e-ac0f-7ec7b47dd447 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:14.196811 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263603d6-d83b-43ae-8fe3-9e1ab34bef4f admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:14.214210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249064b3-e82c-481a-b3d9-d281b53703bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:14.215183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249064b3-e82c-481a-b3d9-d281b53703bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:14.215549 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249064b3-e82c-481a-b3d9-d281b53703bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:14.217304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249064b3-e82c-481a-b3d9-d281b53703bb admin admin] PATH_INFO: `/v3/projects/a8b7dd5d565545d5852cfdd23fea372c` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:14.221695 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249064b3-e82c-481a-b3d9-d281b53703bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8b7dd5d565545d5852cfdd23fea372c)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:14.226982 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249064b3-e82c-481a-b3d9-d281b53703bb admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:14.261511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 871/1766] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:13 2022] POST /identity/v3/projects => generated 387 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:14.263150 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728c6e9-7098-409c-bf05-71da5dd70825 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:14.263652 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728c6e9-7098-409c-bf05-71da5dd70825 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:14.264488 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728c6e9-7098-409c-bf05-71da5dd70825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:14.340743 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-e728c6e9-7098-409c-bf05-71da5dd70825 None None] MFA Rules not processed for user `13128464bb2547f5838a787d022e6c5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:14.342811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e728c6e9-7098-409c-bf05-71da5dd70825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:14.374842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:14.402976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-249064b3-e82c-481a-b3d9-d281b53703bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8b7dd5d565545d5852cfdd23fea372c', 'request_id': 'req-249064b3-e82c-481a-b3d9-d281b53703bb'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:14.435196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-249064b3-e82c-481a-b3d9-d281b53703bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8b7dd5d565545d5852cfdd23fea372c', 'request_id': 'req-249064b3-e82c-481a-b3d9-d281b53703bb'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:14.440193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:14.456816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-249064b3-e82c-481a-b3d9-d281b53703bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8b7dd5d565545d5852cfdd23fea372c', 'request_id': 'req-249064b3-e82c-481a-b3d9-d281b53703bb'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:14.468258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 872/1767] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:14.469254 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728c6e9-7098-409c-bf05-71da5dd70825 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:14.477268 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e728c6e9-7098-409c-bf05-71da5dd70825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:14.533896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-249064b3-e82c-481a-b3d9-d281b53703bb admin admin] The token cache is being invalidate because project a8b7dd5d565545d5852cfdd23fea372c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:14.536061 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-249064b3-e82c-481a-b3d9-d281b53703bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:14.537645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-249064b3-e82c-481a-b3d9-d281b53703bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:14.543231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 896/1768] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:14 2022] DELETE /identity/v3/projects/a8b7dd5d565545d5852cfdd23fea372c => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:14.547627 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c5c76-0f67-41f0-8994-30680d281fde None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:14.548768 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c5c76-0f67-41f0-8994-30680d281fde None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:14.549242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c5c76-0f67-41f0-8994-30680d281fde None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:14.555078 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 897/1769] 10.222.0.65 () {54 vars in 912 bytes} [Wed Aug 3 12:53:14 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:53:14.558049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c5c76-0f67-41f0-8994-30680d281fde None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:14.560719 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0b3c5c76-0f67-41f0-8994-30680d281fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:14.640017 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728c6e9-7098-409c-bf05-71da5dd70825 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:14.654517 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22a5f7-2d56-4546-b31c-3e321e800fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:14.655203 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22a5f7-2d56-4546-b31c-3e321e800fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:14.655498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22a5f7-2d56-4546-b31c-3e321e800fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:14.655761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22a5f7-2d56-4546-b31c-3e321e800fa1 admin admin] PATH_INFO: `/v3/users/20fdf5be1e6848498b07c943466f7183` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:14.657143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22a5f7-2d56-4546-b31c-3e321e800fa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20fdf5be1e6848498b07c943466f7183)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:14.664517 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22a5f7-2d56-4546-b31c-3e321e800fa1 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:14.727372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c5c76-0f67-41f0-8994-30680d281fde None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:14.744137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97d26c-31e7-4290-b6d4-642291170c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:14.746024 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97d26c-31e7-4290-b6d4-642291170c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:14.746565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97d26c-31e7-4290-b6d4-642291170c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:14.747123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97d26c-31e7-4290-b6d4-642291170c46 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:14.748051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97d26c-31e7-4290-b6d4-642291170c46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:14.750891 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa97d26c-31e7-4290-b6d4-642291170c46 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:14.784744 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ba22a5f7-2d56-4546-b31c-3e321e800fa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20fdf5be1e6848498b07c943466f7183'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:14.819703 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 898/1770] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:14 2022] POST /identity/v3/users => generated 300 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:14.820991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97d26c-31e7-4290-b6d4-642291170c46 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:14.824908 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-fa97d26c-31e7-4290-b6d4-642291170c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:14.826957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ba22a5f7-2d56-4546-b31c-3e321e800fa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20fdf5be1e6848498b07c943466f7183'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:14.871187 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ba22a5f7-2d56-4546-b31c-3e321e800fa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20fdf5be1e6848498b07c943466f7183'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:14.882216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ba22a5f7-2d56-4546-b31c-3e321e800fa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20fdf5be1e6848498b07c943466f7183'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:14.911990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97d26c-31e7-4290-b6d4-642291170c46 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:14.930129 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ba22a5f7-2d56-4546-b31c-3e321e800fa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20fdf5be1e6848498b07c943466f7183'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:14.946011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 873/1771] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:14 2022] DELETE /identity/v3/users/20fdf5be1e6848498b07c943466f7183 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:14.949217 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd997c74-411a-4e2d-95b6-c34009d1f102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:14.949874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22a5f7-2d56-4546-b31c-3e321e800fa1 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:14.949981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd997c74-411a-4e2d-95b6-c34009d1f102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:14.950172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd997c74-411a-4e2d-95b6-c34009d1f102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:14.950633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd997c74-411a-4e2d-95b6-c34009d1f102 admin admin] PATH_INFO: `/v3/users/5d699ddb04ce4e4180388aa212d87840` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:14.952879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd997c74-411a-4e2d-95b6-c34009d1f102 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d699ddb04ce4e4180388aa212d87840)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:14.954863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd997c74-411a-4e2d-95b6-c34009d1f102 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:14.955523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ba22a5f7-2d56-4546-b31c-3e321e800fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:15.094433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-dd997c74-411a-4e2d-95b6-c34009d1f102 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d699ddb04ce4e4180388aa212d87840'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:15.128864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-dd997c74-411a-4e2d-95b6-c34009d1f102 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d699ddb04ce4e4180388aa212d87840'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:15.179874 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-dd997c74-411a-4e2d-95b6-c34009d1f102 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d699ddb04ce4e4180388aa212d87840'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:15.182687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22a5f7-2d56-4546-b31c-3e321e800fa1 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:15.185288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-dd997c74-411a-4e2d-95b6-c34009d1f102 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d699ddb04ce4e4180388aa212d87840'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:15.185972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-dd997c74-411a-4e2d-95b6-c34009d1f102 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d699ddb04ce4e4180388aa212d87840'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:15.203626 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac0a7d-fc49-4941-a281-9f1d8473f2ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:15.205500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0a7d-fc49-4941-a281-9f1d8473f2ad service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:15.207375 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0a7d-fc49-4941-a281-9f1d8473f2ad service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:15.208157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac0a7d-fc49-4941-a281-9f1d8473f2ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:15.222911 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f0ac0a7d-fc49-4941-a281-9f1d8473f2ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:15.230054 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 899/1772] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:14 2022] DELETE /identity/v3/users/5d699ddb04ce4e4180388aa212d87840 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:15.239700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd997c74-411a-4e2d-95b6-c34009d1f102 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:15.247480 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-dd997c74-411a-4e2d-95b6-c34009d1f102 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:15.281230 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:15.404988 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:DELETE=14,keystone:SELECT=81,keystone:INSERT=5 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:15.445521 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac0a7d-fc49-4941-a281-9f1d8473f2ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:15.447395 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac0a7d-fc49-4941-a281-9f1d8473f2ad service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:15.471010 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 874/1773] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:15.487358 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032e7ae-051b-4cb8-85e6-93f86324ce50 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:15.487653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032e7ae-051b-4cb8-85e6-93f86324ce50 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:15.487935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032e7ae-051b-4cb8-85e6-93f86324ce50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:15.510874 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd997c74-411a-4e2d-95b6-c34009d1f102 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:15.534289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b836d9d-af40-4d85-ae4c-9da692df7836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:15.535041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b836d9d-af40-4d85-ae4c-9da692df7836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:15.535321 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b836d9d-af40-4d85-ae4c-9da692df7836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:15.535594 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b836d9d-af40-4d85-ae4c-9da692df7836 admin admin] PATH_INFO: `/v3/users/55578411e8c94a1fb4ce8c2a3222a72f` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:15.537601 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b836d9d-af40-4d85-ae4c-9da692df7836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55578411e8c94a1fb4ce8c2a3222a72f)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:15.543102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b836d9d-af40-4d85-ae4c-9da692df7836 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:15.640954 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-e032e7ae-051b-4cb8-85e6-93f86324ce50 None None] MFA Rules not processed for user `52ff458fd7f74b99a745edbcf13a114b`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:15.657189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e032e7ae-051b-4cb8-85e6-93f86324ce50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:15.729928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-7b836d9d-af40-4d85-ae4c-9da692df7836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55578411e8c94a1fb4ce8c2a3222a72f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:15.767292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-7b836d9d-af40-4d85-ae4c-9da692df7836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55578411e8c94a1fb4ce8c2a3222a72f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:15.792673 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 875/1774] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 3 12:53:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:15.793755 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032e7ae-051b-4cb8-85e6-93f86324ce50 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:15.795280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e032e7ae-051b-4cb8-85e6-93f86324ce50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:15.813764 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-7b836d9d-af40-4d85-ae4c-9da692df7836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55578411e8c94a1fb4ce8c2a3222a72f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:15.814888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-7b836d9d-af40-4d85-ae4c-9da692df7836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55578411e8c94a1fb4ce8c2a3222a72f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:15.819211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-7b836d9d-af40-4d85-ae4c-9da692df7836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55578411e8c94a1fb4ce8c2a3222a72f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:15.842816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032e7ae-051b-4cb8-85e6-93f86324ce50 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:15.853340 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cb86a-50e4-4409-a2f8-7426fa5f66c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:15.854309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cb86a-50e4-4409-a2f8-7426fa5f66c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:15.854309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cb86a-50e4-4409-a2f8-7426fa5f66c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:15.860215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8cb86a-50e4-4409-a2f8-7426fa5f66c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:15.860215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cb86a-50e4-4409-a2f8-7426fa5f66c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:15.861531 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8cb86a-50e4-4409-a2f8-7426fa5f66c2 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:15.867806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 900/1775] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:15 2022] DELETE /identity/v3/users/55578411e8c94a1fb4ce8c2a3222a72f => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:15.869592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb251c1-8b27-44a9-81e6-3e84be83ef32 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:15.869958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb251c1-8b27-44a9-81e6-3e84be83ef32 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:15.870666 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb251c1-8b27-44a9-81e6-3e84be83ef32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:15.905472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 876/1776] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:15.907741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cb86a-50e4-4409-a2f8-7426fa5f66c2 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:15.912586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-5f8cb86a-50e4-4409-a2f8-7426fa5f66c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:15.935690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-0fb251c1-8b27-44a9-81e6-3e84be83ef32 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:15.944226 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0fb251c1-8b27-44a9-81e6-3e84be83ef32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:15.948571 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8cb86a-50e4-4409-a2f8-7426fa5f66c2 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:16.138338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 901/1777] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:16.139199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402db79-b528-44ab-a31d-3aed5a4a7a60 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:16.139488 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402db79-b528-44ab-a31d-3aed5a4a7a60 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:16.139763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402db79-b528-44ab-a31d-3aed5a4a7a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:16.143939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f54f91-0522-45de-bea1-d20b2e39e1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:16.145478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f54f91-0522-45de-bea1-d20b2e39e1a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:16.145842 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f54f91-0522-45de-bea1-d20b2e39e1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:16.146199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f54f91-0522-45de-bea1-d20b2e39e1a1 admin admin] PATH_INFO: `/v3/projects/ab36185a7ea44ae5b01e14927a875797/users/c42b23eea5c24109a4b945761c1bf8c8/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:16.172579 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f54f91-0522-45de-bea1-d20b2e39e1a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab36185a7ea44ae5b01e14927a875797, user_id=c42b23eea5c24109a4b945761c1bf8c8, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:16.183217 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f54f91-0522-45de-bea1-d20b2e39e1a1 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:16.212249 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 877/1778] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:15 2022] PUT /identity/v3/projects/ab36185a7ea44ae5b01e14927a875797/users/c42b23eea5c24109a4b945761c1bf8c8/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:16.216225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f54f91-0522-45de-bea1-d20b2e39e1a1 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:16.224214 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-40f54f91-0522-45de-bea1-d20b2e39e1a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:16.239217 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-2402db79-b528-44ab-a31d-3aed5a4a7a60 None None] MFA Rules not processed for user `8aad259ca09340fb99c6b81b4f096dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:16.240485 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-2402db79-b528-44ab-a31d-3aed5a4a7a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:16.284748 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:16.366440 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:16.390283 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f54f91-0522-45de-bea1-d20b2e39e1a1 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:16.401551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae108d-e859-4773-ac07-11b2f29fd8d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:16.402204 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae108d-e859-4773-ac07-11b2f29fd8d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:16.402518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae108d-e859-4773-ac07-11b2f29fd8d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:16.402789 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae108d-e859-4773-ac07-11b2f29fd8d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:16.411517 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-bfae108d-e859-4773-ac07-11b2f29fd8d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:16.432897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 902/1779] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:16.432897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402db79-b528-44ab-a31d-3aed5a4a7a60 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:16.432897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-2402db79-b528-44ab-a31d-3aed5a4a7a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:16.433217 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae108d-e859-4773-ac07-11b2f29fd8d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:16.434309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae108d-e859-4773-ac07-11b2f29fd8d7 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:16.465391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 878/1780] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:16.467197 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae108d-e859-4773-ac07-11b2f29fd8d7 service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:16.469565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-bfae108d-e859-4773-ac07-11b2f29fd8d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:16.488581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae108d-e859-4773-ac07-11b2f29fd8d7 service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:16.493279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2b5bc-6f57-49f9-9a18-fb62dc992294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:16.494183 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2b5bc-6f57-49f9-9a18-fb62dc992294 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:16.494834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2b5bc-6f57-49f9-9a18-fb62dc992294 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:16.495954 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2b5bc-6f57-49f9-9a18-fb62dc992294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:16.593841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402db79-b528-44ab-a31d-3aed5a4a7a60 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:16.609754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516eac4e-0f54-4fb6-99d9-18233d65ca75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:16.610466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516eac4e-0f54-4fb6-99d9-18233d65ca75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:16.610678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516eac4e-0f54-4fb6-99d9-18233d65ca75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:16.610957 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516eac4e-0f54-4fb6-99d9-18233d65ca75 admin admin] PATH_INFO: `/v3/projects/b760a89d7ad8429e88be87f0d44562f9` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:16.612919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516eac4e-0f54-4fb6-99d9-18233d65ca75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b760a89d7ad8429e88be87f0d44562f9)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:16.614897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516eac4e-0f54-4fb6-99d9-18233d65ca75 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:16.645966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2b5bc-6f57-49f9-9a18-fb62dc992294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:16.647610 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2b5bc-6f57-49f9-9a18-fb62dc992294 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:16.666281 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 879/1781] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:16.667289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2b5bc-6f57-49f9-9a18-fb62dc992294 service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:16.684601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2b5bc-6f57-49f9-9a18-fb62dc992294 service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:16.696104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993370eb-aab2-471c-b2d4-586934ba33e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:16.697388 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993370eb-aab2-471c-b2d4-586934ba33e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:16.697829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993370eb-aab2-471c-b2d4-586934ba33e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:16.699161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993370eb-aab2-471c-b2d4-586934ba33e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:16.700064 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993370eb-aab2-471c-b2d4-586934ba33e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:16.704363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993370eb-aab2-471c-b2d4-586934ba33e4 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:16.727097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 880/1782] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:16 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:16.728888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8801e15-7000-49eb-b115-cbd0c1a762c7 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:16.729323 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8801e15-7000-49eb-b115-cbd0c1a762c7 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:16.730201 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8801e15-7000-49eb-b115-cbd0c1a762c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:16.750365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-516eac4e-0f54-4fb6-99d9-18233d65ca75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b760a89d7ad8429e88be87f0d44562f9', 'request_id': 'req-516eac4e-0f54-4fb6-99d9-18233d65ca75'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:16.760854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-e8801e15-7000-49eb-b115-cbd0c1a762c7 None None] MFA Rules not processed for user `8aad259ca09340fb99c6b81b4f096dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:16.763141 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e8801e15-7000-49eb-b115-cbd0c1a762c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:16.778371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-516eac4e-0f54-4fb6-99d9-18233d65ca75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b760a89d7ad8429e88be87f0d44562f9', 'request_id': 'req-516eac4e-0f54-4fb6-99d9-18233d65ca75'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:16.791554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-516eac4e-0f54-4fb6-99d9-18233d65ca75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b760a89d7ad8429e88be87f0d44562f9', 'request_id': 'req-516eac4e-0f54-4fb6-99d9-18233d65ca75'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:16.812813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-516eac4e-0f54-4fb6-99d9-18233d65ca75 admin admin] The token cache is being invalidate because project b760a89d7ad8429e88be87f0d44562f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:16.813130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-516eac4e-0f54-4fb6-99d9-18233d65ca75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:16.821859 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-516eac4e-0f54-4fb6-99d9-18233d65ca75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:16.823711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 903/1783] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:16 2022] DELETE /identity/v3/projects/b760a89d7ad8429e88be87f0d44562f9 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:16.824504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516eac4e-0f54-4fb6-99d9-18233d65ca75 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:16.831403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-516eac4e-0f54-4fb6-99d9-18233d65ca75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:16.899610 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 881/1784] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:16.901159 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e223f92-be65-4ef8-a66d-f68b064ef275 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:16.901517 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e223f92-be65-4ef8-a66d-f68b064ef275 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:16.901883 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e223f92-be65-4ef8-a66d-f68b064ef275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:16.931041 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-1e223f92-be65-4ef8-a66d-f68b064ef275 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:16.932436 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1e223f92-be65-4ef8-a66d-f68b064ef275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:16.965957 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516eac4e-0f54-4fb6-99d9-18233d65ca75 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:16.977132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b16404-d577-4b16-9402-60802d05f68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:16.978114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b16404-d577-4b16-9402-60802d05f68d admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:16.978723 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b16404-d577-4b16-9402-60802d05f68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:16.979156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b16404-d577-4b16-9402-60802d05f68d admin admin] PATH_INFO: `/v3/projects/ab36185a7ea44ae5b01e14927a875797/users/c42b23eea5c24109a4b945761c1bf8c8/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:16.986672 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b16404-d577-4b16-9402-60802d05f68d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab36185a7ea44ae5b01e14927a875797, user_id=c42b23eea5c24109a4b945761c1bf8c8, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:16.990222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b16404-d577-4b16-9402-60802d05f68d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:17.015430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 904/1785] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:16 2022] PUT /identity/v3/projects/ab36185a7ea44ae5b01e14927a875797/users/c42b23eea5c24109a4b945761c1bf8c8/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:17.016210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b16404-d577-4b16-9402-60802d05f68d admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:17.024047 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d0b16404-d577-4b16-9402-60802d05f68d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:17.028129 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:17.044415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:17.070082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 882/1786] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:17.072659 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f4065-a06a-4e53-b281-94d4c6284783 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:17.074090 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f4065-a06a-4e53-b281-94d4c6284783 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:17.074377 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f4065-a06a-4e53-b281-94d4c6284783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:17.155395 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-7c5f4065-a06a-4e53-b281-94d4c6284783 None None] MFA Rules not processed for user `c42b23eea5c24109a4b945761c1bf8c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:17.157284 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-7c5f4065-a06a-4e53-b281-94d4c6284783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:17.189878 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b16404-d577-4b16-9402-60802d05f68d admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:17.199369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3290d07-635c-4810-9fd2-ac2c15968f0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:17.200025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3290d07-635c-4810-9fd2-ac2c15968f0a service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:17.200308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3290d07-635c-4810-9fd2-ac2c15968f0a service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:17.200587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3290d07-635c-4810-9fd2-ac2c15968f0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:17.212109 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a3290d07-635c-4810-9fd2-ac2c15968f0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:17.388454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3290d07-635c-4810-9fd2-ac2c15968f0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:17.394624 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3290d07-635c-4810-9fd2-ac2c15968f0a service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:17.429524 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 905/1787] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:17.431564 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa767273-bf0f-47b9-bca5-e1fddd7971fe None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:17.434414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa767273-bf0f-47b9-bca5-e1fddd7971fe None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:17.434977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa767273-bf0f-47b9-bca5-e1fddd7971fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:17.461690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 883/1788] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:17 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:17.466640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f4065-a06a-4e53-b281-94d4c6284783 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:17.468536 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-7c5f4065-a06a-4e53-b281-94d4c6284783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:17.492806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-fa767273-bf0f-47b9-bca5-e1fddd7971fe None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:17.494065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-fa767273-bf0f-47b9-bca5-e1fddd7971fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:17.622403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f4065-a06a-4e53-b281-94d4c6284783 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:17.625503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 906/1789] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:17.628242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa767273-bf0f-47b9-bca5-e1fddd7971fe None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:17.630563 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-fa767273-bf0f-47b9-bca5-e1fddd7971fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:17.633935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762adc4d-852a-498d-9a9c-9efb9646eb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:17.634567 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762adc4d-852a-498d-9a9c-9efb9646eb91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:17.635190 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762adc4d-852a-498d-9a9c-9efb9646eb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:17.635190 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762adc4d-852a-498d-9a9c-9efb9646eb91 admin admin] PATH_INFO: `/v3/projects/d65297d6db9641f6bdfe47ef79b657d5` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:17.644794 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762adc4d-852a-498d-9a9c-9efb9646eb91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d65297d6db9641f6bdfe47ef79b657d5)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:17.646884 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762adc4d-852a-498d-9a9c-9efb9646eb91 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:17.759888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:17.769124 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa767273-bf0f-47b9-bca5-e1fddd7971fe None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:17.773750 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-762adc4d-852a-498d-9a9c-9efb9646eb91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd65297d6db9641f6bdfe47ef79b657d5', 'request_id': 'req-762adc4d-852a-498d-9a9c-9efb9646eb91'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:17.797049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-762adc4d-852a-498d-9a9c-9efb9646eb91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd65297d6db9641f6bdfe47ef79b657d5', 'request_id': 'req-762adc4d-852a-498d-9a9c-9efb9646eb91'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:17.805311 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-762adc4d-852a-498d-9a9c-9efb9646eb91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd65297d6db9641f6bdfe47ef79b657d5', 'request_id': 'req-762adc4d-852a-498d-9a9c-9efb9646eb91'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:17.827057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-762adc4d-852a-498d-9a9c-9efb9646eb91 admin admin] The token cache is being invalidate because project d65297d6db9641f6bdfe47ef79b657d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:17.827361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-762adc4d-852a-498d-9a9c-9efb9646eb91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:17.828099 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-762adc4d-852a-498d-9a9c-9efb9646eb91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:17.834477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 884/1790] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:17 2022] DELETE /identity/v3/projects/d65297d6db9641f6bdfe47ef79b657d5 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:17.834477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762adc4d-852a-498d-9a9c-9efb9646eb91 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:17.834477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-762adc4d-852a-498d-9a9c-9efb9646eb91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:17.848624 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9864b82-e911-4a80-8c18-73521c232350 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:17.849406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:17.849914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9864b82-e911-4a80-8c18-73521c232350 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:17.850375 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9864b82-e911-4a80-8c18-73521c232350 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:17.851355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9864b82-e911-4a80-8c18-73521c232350 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:17.854720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e9864b82-e911-4a80-8c18-73521c232350 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:17.966249 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762adc4d-852a-498d-9a9c-9efb9646eb91 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:17.971045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9864b82-e911-4a80-8c18-73521c232350 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:17.973987 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3620b7d-6b26-4839-b1ec-6ed19744e7ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5094263ac471481db48783b19192e01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:17.974558 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9864b82-e911-4a80-8c18-73521c232350 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:17.975311 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3620b7d-6b26-4839-b1ec-6ed19744e7ff service placement] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:17.975642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3620b7d-6b26-4839-b1ec-6ed19744e7ff service placement] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:17.977238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3620b7d-6b26-4839-b1ec-6ed19744e7ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:17.979299 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c3620b7d-6b26-4839-b1ec-6ed19744e7ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:17.988208 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 907/1791] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:17.989442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9864b82-e911-4a80-8c18-73521c232350 service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:17.992373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e9864b82-e911-4a80-8c18-73521c232350 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:18.000634 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3620b7d-6b26-4839-b1ec-6ed19744e7ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:18.004110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3620b7d-6b26-4839-b1ec-6ed19744e7ff service placement] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:18.035690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 885/1792] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:18.038368 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1543fb-6326-4bc3-ac12-1673f9e09223 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:18.038368 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1543fb-6326-4bc3-ac12-1673f9e09223 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:18.038368 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1543fb-6326-4bc3-ac12-1673f9e09223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:18.091446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-6a1543fb-6326-4bc3-ac12-1673f9e09223 None None] MFA Rules not processed for user `44fbd50cb5664149814abf0e9e44e18f`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:18.093276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6a1543fb-6326-4bc3-ac12-1673f9e09223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:18.121299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9864b82-e911-4a80-8c18-73521c232350 service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:18.135626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec864b2-3f53-45d9-b6ae-537c84a0ab95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:18.142580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec864b2-3f53-45d9-b6ae-537c84a0ab95 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:18.142580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec864b2-3f53-45d9-b6ae-537c84a0ab95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:18.142580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec864b2-3f53-45d9-b6ae-537c84a0ab95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:18.142580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec864b2-3f53-45d9-b6ae-537c84a0ab95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:53:18.142580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec864b2-3f53-45d9-b6ae-537c84a0ab95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:18.142580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec864b2-3f53-45d9-b6ae-537c84a0ab95 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:18.199377 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 908/1793] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:53:18.201592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f689db1-e12d-4911-8f00-3de293333042 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:18.203255 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f689db1-e12d-4911-8f00-3de293333042 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:18.203782 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f689db1-e12d-4911-8f00-3de293333042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:18.254144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1f689db1-e12d-4911-8f00-3de293333042 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:18.256649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1f689db1-e12d-4911-8f00-3de293333042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:18.282550 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 886/1794] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:18.283320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1543fb-6326-4bc3-ac12-1673f9e09223 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:18.286945 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 909/1795] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:18.296368 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e81680-35a9-4bec-865d-1bf0da01788c None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:18.298060 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e81680-35a9-4bec-865d-1bf0da01788c None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:18.298554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e81680-35a9-4bec-865d-1bf0da01788c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:18.347642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1543fb-6326-4bc3-ac12-1673f9e09223 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:18.372393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-58e81680-35a9-4bec-865d-1bf0da01788c None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:18.380878 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-58e81680-35a9-4bec-865d-1bf0da01788c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:18.382388 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2182d-29d9-4312-bcae-b82e165381fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:18.383056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2182d-29d9-4312-bcae-b82e165381fc admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:18.383334 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2182d-29d9-4312-bcae-b82e165381fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:18.383605 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2182d-29d9-4312-bcae-b82e165381fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:18.384380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2182d-29d9-4312-bcae-b82e165381fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:18.387505 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab2182d-29d9-4312-bcae-b82e165381fc admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:18.406812 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 910/1796] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:18.416317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e81680-35a9-4bec-865d-1bf0da01788c None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:18.439645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e81680-35a9-4bec-865d-1bf0da01788c None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:18.461832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729122bf-ec92-45ff-aaca-5121657aa42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:18.462517 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729122bf-ec92-45ff-aaca-5121657aa42b admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:18.462790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729122bf-ec92-45ff-aaca-5121657aa42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:18.463061 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729122bf-ec92-45ff-aaca-5121657aa42b admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:18.463784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729122bf-ec92-45ff-aaca-5121657aa42b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:53:18.464083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729122bf-ec92-45ff-aaca-5121657aa42b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:18.465749 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729122bf-ec92-45ff-aaca-5121657aa42b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:18.505615 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 911/1797] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:53:18.510260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729122bf-ec92-45ff-aaca-5121657aa42b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:18.517359 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 887/1798] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:18 2022] POST /identity/v3/projects => generated 377 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:18.518026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-729122bf-ec92-45ff-aaca-5121657aa42b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:18.520703 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2182d-29d9-4312-bcae-b82e165381fc admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:18.572315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729122bf-ec92-45ff-aaca-5121657aa42b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:18.618449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:18.769506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a2547-21fa-4580-ac16-789088f37489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:18.770317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a2547-21fa-4580-ac16-789088f37489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:18.770680 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a2547-21fa-4580-ac16-789088f37489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:18.770961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a2547-21fa-4580-ac16-789088f37489 admin admin] PATH_INFO: `/v3/projects/24d2205a09d64fefad6e0d890544dab4` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:18.773843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a2547-21fa-4580-ac16-789088f37489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24d2205a09d64fefad6e0d890544dab4)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:18.776101 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a2547-21fa-4580-ac16-789088f37489 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:18.825009 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2182d-29d9-4312-bcae-b82e165381fc admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:18.838026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19c5ab-ce97-4573-bc0b-79c0f422cfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:18.838636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19c5ab-ce97-4573-bc0b-79c0f422cfc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:18.838919 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19c5ab-ce97-4573-bc0b-79c0f422cfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:18.839188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19c5ab-ce97-4573-bc0b-79c0f422cfc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:18.842354 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19c5ab-ce97-4573-bc0b-79c0f422cfc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:18.846893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19c5ab-ce97-4573-bc0b-79c0f422cfc1 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:18.927157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5f2a2547-21fa-4580-ac16-789088f37489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24d2205a09d64fefad6e0d890544dab4', 'request_id': 'req-5f2a2547-21fa-4580-ac16-789088f37489'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:18.930903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 888/1799] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:18 2022] POST /identity/v3/projects => generated 377 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:18.932457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19c5ab-ce97-4573-bc0b-79c0f422cfc1 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:18.943166 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:18.947815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5f2a2547-21fa-4580-ac16-789088f37489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24d2205a09d64fefad6e0d890544dab4', 'request_id': 'req-5f2a2547-21fa-4580-ac16-789088f37489'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:18.976769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5f2a2547-21fa-4580-ac16-789088f37489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24d2205a09d64fefad6e0d890544dab4', 'request_id': 'req-5f2a2547-21fa-4580-ac16-789088f37489'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:19.011526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5f2a2547-21fa-4580-ac16-789088f37489 admin admin] The token cache is being invalidate because project 24d2205a09d64fefad6e0d890544dab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:19.012005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5f2a2547-21fa-4580-ac16-789088f37489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:19.015369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5f2a2547-21fa-4580-ac16-789088f37489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:19.017914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 912/1800] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:18 2022] DELETE /identity/v3/projects/24d2205a09d64fefad6e0d890544dab4 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:19.020005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ffb80-6ab8-48c8-ba45-c86c948a05cd None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:19.021977 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19c5ab-ce97-4573-bc0b-79c0f422cfc1 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:19.022437 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ffb80-6ab8-48c8-ba45-c86c948a05cd None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:19.024682 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ffb80-6ab8-48c8-ba45-c86c948a05cd None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:19.027332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec19ed2-d272-4f82-a060-445b5999d964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:19.027854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 913/1801] 10.222.0.65 () {54 vars in 912 bytes} [Wed Aug 3 12:53:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:53:19.028020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec19ed2-d272-4f82-a060-445b5999d964 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:19.028313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec19ed2-d272-4f82-a060-445b5999d964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:19.028589 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec19ed2-d272-4f82-a060-445b5999d964 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:19.029265 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec19ed2-d272-4f82-a060-445b5999d964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:19.030366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a9a59-2354-4fb2-a6b9-c205aa12d00a None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:19.031930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec19ed2-d272-4f82-a060-445b5999d964 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:19.032049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a9a59-2354-4fb2-a6b9-c205aa12d00a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:19.032498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a9a59-2354-4fb2-a6b9-c205aa12d00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:19.071886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-991a9a59-2354-4fb2-a6b9-c205aa12d00a None None] MFA Rules not processed for user `5f16ff28aac54514a475f8a4f5c6e4c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:19.072783 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 889/1802] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:18 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:19.073890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec19ed2-d272-4f82-a060-445b5999d964 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:19.075438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-991a9a59-2354-4fb2-a6b9-c205aa12d00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:19.076912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-3ec19ed2-d272-4f82-a060-445b5999d964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:19.099988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec19ed2-d272-4f82-a060-445b5999d964 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:19.108814 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c8a287-e264-4e66-bfbd-8dbf01bfe63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:19.109566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8a287-e264-4e66-bfbd-8dbf01bfe63e admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:19.109855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8a287-e264-4e66-bfbd-8dbf01bfe63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:19.110178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8a287-e264-4e66-bfbd-8dbf01bfe63e admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:19.110965 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8a287-e264-4e66-bfbd-8dbf01bfe63e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:19.113724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c8a287-e264-4e66-bfbd-8dbf01bfe63e admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:19.172351 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 890/1803] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:19 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:19.175377 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc45e32-5c52-42b0-833d-43531f261ff1 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:19.175667 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc45e32-5c52-42b0-833d-43531f261ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:19.175944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc45e32-5c52-42b0-833d-43531f261ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:19.230264 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-5dc45e32-5c52-42b0-833d-43531f261ff1 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:19.232398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-5dc45e32-5c52-42b0-833d-43531f261ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:19.255138 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 891/1804] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:19.258840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff5f92-4469-437f-9119-c6c830070254 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:19.259130 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff5f92-4469-437f-9119-c6c830070254 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:19.259407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff5f92-4469-437f-9119-c6c830070254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:19.293135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 914/1805] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:19.295353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a9a59-2354-4fb2-a6b9-c205aa12d00a None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:19.299217 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-991a9a59-2354-4fb2-a6b9-c205aa12d00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:19.305131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-4bff5f92-4469-437f-9119-c6c830070254 None None] MFA Rules not processed for user `52ff458fd7f74b99a745edbcf13a114b`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:19.309329 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-4bff5f92-4469-437f-9119-c6c830070254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:19.339350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a9a59-2354-4fb2-a6b9-c205aa12d00a None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:19.363025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00843f-9d54-4dff-9664-49a13c2aeb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:19.367130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00843f-9d54-4dff-9664-49a13c2aeb6c admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:19.367511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00843f-9d54-4dff-9664-49a13c2aeb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:19.367802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00843f-9d54-4dff-9664-49a13c2aeb6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:19.368538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00843f-9d54-4dff-9664-49a13c2aeb6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:19.375781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00843f-9d54-4dff-9664-49a13c2aeb6c admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:19.411396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 915/1806] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:19.414573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00843f-9d54-4dff-9664-49a13c2aeb6c admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:19.423794 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 892/1807] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 3 12:53:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:19.427139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adfab2-e050-4d8a-a94e-3604921a061f None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:19.427816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adfab2-e050-4d8a-a94e-3604921a061f None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:19.428102 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22adfab2-e050-4d8a-a94e-3604921a061f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:19.443943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00843f-9d54-4dff-9664-49a13c2aeb6c admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:19.450733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-22adfab2-e050-4d8a-a94e-3604921a061f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:19.452218 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-22adfab2-e050-4d8a-a94e-3604921a061f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:19.462957 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0858c9-3e0f-4d27-8d2a-670bdd0a4df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:19.463585 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0858c9-3e0f-4d27-8d2a-670bdd0a4df7 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:19.463918 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0858c9-3e0f-4d27-8d2a-670bdd0a4df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:19.464239 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0858c9-3e0f-4d27-8d2a-670bdd0a4df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:19.464899 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0858c9-3e0f-4d27-8d2a-670bdd0a4df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:19.467029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0858c9-3e0f-4d27-8d2a-670bdd0a4df7 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:19.468987 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 893/1808] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:19.470084 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22adfab2-e050-4d8a-a94e-3604921a061f None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:19.471696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-22adfab2-e050-4d8a-a94e-3604921a061f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:19.485846 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 916/1809] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:19.487701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0858c9-3e0f-4d27-8d2a-670bdd0a4df7 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:19.488913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22adfab2-e050-4d8a-a94e-3604921a061f None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:19.494615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf03622-db21-41e4-8c76-7e9c497e69ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:19.495431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf03622-db21-41e4-8c76-7e9c497e69ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:19.495769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf03622-db21-41e4-8c76-7e9c497e69ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:19.496067 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf03622-db21-41e4-8c76-7e9c497e69ac admin admin] PATH_INFO: `/v3/users/9aee697b6b97434da6b4324dad02f5b2` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:19.497527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf03622-db21-41e4-8c76-7e9c497e69ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aee697b6b97434da6b4324dad02f5b2)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:19.499156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0858c9-3e0f-4d27-8d2a-670bdd0a4df7 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:19.499866 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf03622-db21-41e4-8c76-7e9c497e69ac admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:19.505306 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee503c45-c93c-4318-a8ae-5733ce4cc437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:19.505932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee503c45-c93c-4318-a8ae-5733ce4cc437 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:19.506212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee503c45-c93c-4318-a8ae-5733ce4cc437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:19.506630 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee503c45-c93c-4318-a8ae-5733ce4cc437 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:19.507738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee503c45-c93c-4318-a8ae-5733ce4cc437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:19.509328 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee503c45-c93c-4318-a8ae-5733ce4cc437 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:19.528373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 917/1810] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:19.528373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee503c45-c93c-4318-a8ae-5733ce4cc437 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:19.529554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ee503c45-c93c-4318-a8ae-5733ce4cc437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:19.544925 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee503c45-c93c-4318-a8ae-5733ce4cc437 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:19.550978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87220213-0add-431d-8863-896311a1905e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:19.551859 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87220213-0add-431d-8863-896311a1905e service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:19.552171 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87220213-0add-431d-8863-896311a1905e service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:19.552669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87220213-0add-431d-8863-896311a1905e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:19.564866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87220213-0add-431d-8863-896311a1905e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:19.568479 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87220213-0add-431d-8863-896311a1905e service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:19.582638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 918/1811] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:19.583406 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87220213-0add-431d-8863-896311a1905e service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:19.599407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87220213-0add-431d-8863-896311a1905e service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:19.602536 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8bf03622-db21-41e4-8c76-7e9c497e69ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aee697b6b97434da6b4324dad02f5b2'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:19.607680 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88c995-cf5b-4c39-918a-4f2f0ad4addb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:19.608542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88c995-cf5b-4c39-918a-4f2f0ad4addb admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:19.608937 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88c995-cf5b-4c39-918a-4f2f0ad4addb admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:19.609818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88c995-cf5b-4c39-918a-4f2f0ad4addb admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:19.611311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f88c995-cf5b-4c39-918a-4f2f0ad4addb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:53:19.611884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f88c995-cf5b-4c39-918a-4f2f0ad4addb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:19.614370 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f88c995-cf5b-4c39-918a-4f2f0ad4addb admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:19.620696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:19.634412 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8bf03622-db21-41e4-8c76-7e9c497e69ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aee697b6b97434da6b4324dad02f5b2'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:19.641472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=84,keystone:INSERT=7,keystone:DELETE=9 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:19.642682 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 919/1812] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:53:19.648833 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88c995-cf5b-4c39-918a-4f2f0ad4addb admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:19.659580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8bf03622-db21-41e4-8c76-7e9c497e69ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aee697b6b97434da6b4324dad02f5b2'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:19.660755 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8bf03622-db21-41e4-8c76-7e9c497e69ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aee697b6b97434da6b4324dad02f5b2'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:19.674930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8bf03622-db21-41e4-8c76-7e9c497e69ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aee697b6b97434da6b4324dad02f5b2'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:19.696733 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88c995-cf5b-4c39-918a-4f2f0ad4addb admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:19.700142 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 894/1813] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:19 2022] DELETE /identity/v3/users/9aee697b6b97434da6b4324dad02f5b2 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:19.701299 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf03622-db21-41e4-8c76-7e9c497e69ac admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:19.708347 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6e722-e755-46a3-8a79-e591058aa236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:19.708768 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8bf03622-db21-41e4-8c76-7e9c497e69ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:19.709481 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6e722-e755-46a3-8a79-e591058aa236 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:19.710178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6e722-e755-46a3-8a79-e591058aa236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:19.710741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6e722-e755-46a3-8a79-e591058aa236 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:19.711620 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6e722-e755-46a3-8a79-e591058aa236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:19.714955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6e722-e755-46a3-8a79-e591058aa236 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:19.733440 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 920/1814] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:19.734805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6e722-e755-46a3-8a79-e591058aa236 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:19.740611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e5f6e722-e755-46a3-8a79-e591058aa236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:19.763365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6e722-e755-46a3-8a79-e591058aa236 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:19.960471 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb26c13-a0b8-4998-a0d5-f0a0a959c476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:19.961297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb26c13-a0b8-4998-a0d5-f0a0a959c476 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:19.961678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb26c13-a0b8-4998-a0d5-f0a0a959c476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:19.962065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb26c13-a0b8-4998-a0d5-f0a0a959c476 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:19.963746 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb26c13-a0b8-4998-a0d5-f0a0a959c476 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:19.965850 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb26c13-a0b8-4998-a0d5-f0a0a959c476 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:19.971164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf03622-db21-41e4-8c76-7e9c497e69ac admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:19.993985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fc066-e280-4593-8b56-ad073676a708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:20.004389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fc066-e280-4593-8b56-ad073676a708 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:20.004960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fc066-e280-4593-8b56-ad073676a708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:20.005509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fc066-e280-4593-8b56-ad073676a708 admin admin] PATH_INFO: `/v3/projects/4ae6edf63ed64a6fa7fb3ee8b443f253/users/d7bf4677b7c04cd687673f01de256ff1/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:20.031566 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 921/1815] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:19 2022] POST /identity/v3/projects => generated 377 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:20.044494 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084e99-b606-4b98-a305-1c255c793f04 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:20.047949 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084e99-b606-4b98-a305-1c255c793f04 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:20.048368 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a084e99-b606-4b98-a305-1c255c793f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:20.056685 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fc066-e280-4593-8b56-ad073676a708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ae6edf63ed64a6fa7fb3ee8b443f253, user_id=d7bf4677b7c04cd687673f01de256ff1, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:20.059719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104fc066-e280-4593-8b56-ad073676a708 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:20.109412 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 895/1816] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:19 2022] PUT /identity/v3/projects/4ae6edf63ed64a6fa7fb3ee8b443f253/users/d7bf4677b7c04cd687673f01de256ff1/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:20.109412 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fc066-e280-4593-8b56-ad073676a708 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:20.109412 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-104fc066-e280-4593-8b56-ad073676a708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:20.119387 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-4a084e99-b606-4b98-a305-1c255c793f04 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:20.121280 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-4a084e99-b606-4b98-a305-1c255c793f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:20.256132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 922/1817] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:20.259783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a084e99-b606-4b98-a305-1c255c793f04 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:20.282467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a084e99-b606-4b98-a305-1c255c793f04 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:20.294278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4140ef4-2e4b-4fef-9abb-bdb3ddc8020f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:20.295109 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4140ef4-2e4b-4fef-9abb-bdb3ddc8020f admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:20.297823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4140ef4-2e4b-4fef-9abb-bdb3ddc8020f admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:20.298197 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4140ef4-2e4b-4fef-9abb-bdb3ddc8020f admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:20.299215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4140ef4-2e4b-4fef-9abb-bdb3ddc8020f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:20.300923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fc066-e280-4593-8b56-ad073676a708 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:20.306471 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d76995-cd1c-4101-be7d-6170ecc94e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:20.307397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4140ef4-2e4b-4fef-9abb-bdb3ddc8020f admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:20.307496 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d76995-cd1c-4101-be7d-6170ecc94e26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:20.307892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d76995-cd1c-4101-be7d-6170ecc94e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:20.308244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d76995-cd1c-4101-be7d-6170ecc94e26 admin admin] PATH_INFO: `/v3/projects/48ad2f7d710548259e5ddcf70a7d213b/users/4eb0a3d0aada4db1a80f19a7267e490b/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:20.331799 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d76995-cd1c-4101-be7d-6170ecc94e26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48ad2f7d710548259e5ddcf70a7d213b, user_id=4eb0a3d0aada4db1a80f19a7267e490b, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:20.349015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d76995-cd1c-4101-be7d-6170ecc94e26 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:20.369877 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 923/1818] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:20 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:20.378210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578d15d-0e31-43bf-978d-0338d84c8ef1 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:20.378588 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578d15d-0e31-43bf-978d-0338d84c8ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:20.378869 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578d15d-0e31-43bf-978d-0338d84c8ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:20.398115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 896/1819] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:20 2022] PUT /identity/v3/projects/48ad2f7d710548259e5ddcf70a7d213b/users/4eb0a3d0aada4db1a80f19a7267e490b/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:20.399342 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d76995-cd1c-4101-be7d-6170ecc94e26 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:20.401678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-07d76995-cd1c-4101-be7d-6170ecc94e26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:20.469652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1578d15d-0e31-43bf-978d-0338d84c8ef1 None None] MFA Rules not processed for user `d7bf4677b7c04cd687673f01de256ff1`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:20.471149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1578d15d-0e31-43bf-978d-0338d84c8ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:20.591316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d76995-cd1c-4101-be7d-6170ecc94e26 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:20.597644 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579aaf66-44dd-4479-9362-f72dade782f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:20.598389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aaf66-44dd-4479-9362-f72dade782f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:20.598988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aaf66-44dd-4479-9362-f72dade782f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:20.599317 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aaf66-44dd-4479-9362-f72dade782f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:20.606969 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:20.616667 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:20.682336 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 924/1820] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:20.682336 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578d15d-0e31-43bf-978d-0338d84c8ef1 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:20.767001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579aaf66-44dd-4479-9362-f72dade782f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:20.769206 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579aaf66-44dd-4479-9362-f72dade782f0 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:20.782785 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 897/1821] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:20.784393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd903c-18b9-4da8-ad6d-2237fc7c5077 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:20.784894 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd903c-18b9-4da8-ad6d-2237fc7c5077 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:20.785404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cd903c-18b9-4da8-ad6d-2237fc7c5077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:20.837645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-71cd903c-18b9-4da8-ad6d-2237fc7c5077 None None] MFA Rules not processed for user `4eb0a3d0aada4db1a80f19a7267e490b`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:20.839010 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-71cd903c-18b9-4da8-ad6d-2237fc7c5077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:20.951226 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578d15d-0e31-43bf-978d-0338d84c8ef1 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:20.970842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eee321-1505-4330-be4c-d64ac1d2deb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:20.972157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eee321-1505-4330-be4c-d64ac1d2deb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:20.974882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eee321-1505-4330-be4c-d64ac1d2deb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:20.975267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eee321-1505-4330-be4c-d64ac1d2deb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:20.976080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eee321-1505-4330-be4c-d64ac1d2deb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:20.978982 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eee321-1505-4330-be4c-d64ac1d2deb7 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:21.001864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 898/1822] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:21.002791 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd903c-18b9-4da8-ad6d-2237fc7c5077 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:21.011540 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 925/1823] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:20 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:21.012351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eee321-1505-4330-be4c-d64ac1d2deb7 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:21.025814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eee321-1505-4330-be4c-d64ac1d2deb7 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:21.033396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c2e7c-c7f6-4d61-a3d2-5be899aa8a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:21.035029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c2e7c-c7f6-4d61-a3d2-5be899aa8a3e admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:21.035029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c2e7c-c7f6-4d61-a3d2-5be899aa8a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:21.035218 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c2e7c-c7f6-4d61-a3d2-5be899aa8a3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:21.035900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c2e7c-c7f6-4d61-a3d2-5be899aa8a3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:21.039199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c2e7c-c7f6-4d61-a3d2-5be899aa8a3e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:21.049567 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cd903c-18b9-4da8-ad6d-2237fc7c5077 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:21.064467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d386d-0ec7-4ec9-90f1-1b8b37bd26c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:21.065701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d386d-0ec7-4ec9-90f1-1b8b37bd26c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:21.065701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d386d-0ec7-4ec9-90f1-1b8b37bd26c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:21.065701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d386d-0ec7-4ec9-90f1-1b8b37bd26c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:21.066384 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d386d-0ec7-4ec9-90f1-1b8b37bd26c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:21.071042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d386d-0ec7-4ec9-90f1-1b8b37bd26c9 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:21.119319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 926/1824] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:21 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:21.120686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c2e7c-c7f6-4d61-a3d2-5be899aa8a3e admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:21.168237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 899/1825] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:21 2022] POST /identity/v3/projects => generated 375 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:21.169940 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d386d-0ec7-4ec9-90f1-1b8b37bd26c9 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:21.302947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c2e7c-c7f6-4d61-a3d2-5be899aa8a3e admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:21.404966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d386d-0ec7-4ec9-90f1-1b8b37bd26c9 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:21.415825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5ee78-90ed-44e7-8fa3-adb148f052c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:21.416616 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5ee78-90ed-44e7-8fa3-adb148f052c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:21.416896 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5ee78-90ed-44e7-8fa3-adb148f052c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:21.417192 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5ee78-90ed-44e7-8fa3-adb148f052c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:21.417829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5ee78-90ed-44e7-8fa3-adb148f052c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:21.423593 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5ee78-90ed-44e7-8fa3-adb148f052c4 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:21.471518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:21.487012 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 900/1826] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:21 2022] POST /identity/v3/users => generated 290 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:21.488063 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5ee78-90ed-44e7-8fa3-adb148f052c4 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:21.500470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5ee78-90ed-44e7-8fa3-adb148f052c4 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:21.501778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fdc62-c104-48a8-92a9-4c44ccb0022c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:21.503193 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fdc62-c104-48a8-92a9-4c44ccb0022c admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:21.503542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fdc62-c104-48a8-92a9-4c44ccb0022c admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:21.503823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fdc62-c104-48a8-92a9-4c44ccb0022c admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:21.504738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fdc62-c104-48a8-92a9-4c44ccb0022c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:21.506367 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fdc62-c104-48a8-92a9-4c44ccb0022c admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:21.506642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0a8b6-5dc8-476b-826a-7f131de66153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:21.507468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0a8b6-5dc8-476b-826a-7f131de66153 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:21.508048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0a8b6-5dc8-476b-826a-7f131de66153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:21.508471 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0a8b6-5dc8-476b-826a-7f131de66153 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:21.509862 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0a8b6-5dc8-476b-826a-7f131de66153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:21.513360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0a8b6-5dc8-476b-826a-7f131de66153 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:21.538306 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:21.556660 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 927/1827] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 437 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:21.559903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 901/1828] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:21 2022] POST /identity/v3/users => generated 294 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:21.560021 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fdc62-c104-48a8-92a9-4c44ccb0022c admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:21.562035 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0a8b6-5dc8-476b-826a-7f131de66153 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:21.590730 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fdc62-c104-48a8-92a9-4c44ccb0022c admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:21.600640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0a8b6-5dc8-476b-826a-7f131de66153 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:21.601877 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e9ecb-92f3-4c4c-9bba-459a5b4c4f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:21.603320 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e9ecb-92f3-4c4c-9bba-459a5b4c4f7d admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:21.603646 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e9ecb-92f3-4c4c-9bba-459a5b4c4f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:21.603925 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e9ecb-92f3-4c4c-9bba-459a5b4c4f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:21.604754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e9ecb-92f3-4c4c-9bba-459a5b4c4f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:21.607723 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e9ecb-92f3-4c4c-9bba-459a5b4c4f7d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:21.619275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b94247-1fa3-4685-b7b6-a23a4edb6549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:21.621129 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b94247-1fa3-4685-b7b6-a23a4edb6549 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:21.622077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b94247-1fa3-4685-b7b6-a23a4edb6549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:21.622363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b94247-1fa3-4685-b7b6-a23a4edb6549 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:21.625022 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b94247-1fa3-4685-b7b6-a23a4edb6549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:21.627729 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b94247-1fa3-4685-b7b6-a23a4edb6549 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:21.651050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 928/1829] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:21.651836 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e9ecb-92f3-4c4c-9bba-459a5b4c4f7d admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:21.667633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 902/1830] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:21.668720 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b94247-1fa3-4685-b7b6-a23a4edb6549 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:21.682721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b94247-1fa3-4685-b7b6-a23a4edb6549 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:21.689502 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3549d45b-2b5d-48b2-858a-4ed9c2d49042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:21.691624 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549d45b-2b5d-48b2-858a-4ed9c2d49042 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:21.691912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549d45b-2b5d-48b2-858a-4ed9c2d49042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:21.692435 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549d45b-2b5d-48b2-858a-4ed9c2d49042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:21.693176 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549d45b-2b5d-48b2-858a-4ed9c2d49042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:21.695538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3549d45b-2b5d-48b2-858a-4ed9c2d49042 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:21.708417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e9ecb-92f3-4c4c-9bba-459a5b4c4f7d admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:21.710220 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 903/1831] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:21.715192 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa76f9-a1a9-4b52-95c4-648793a021b0 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:21.715486 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa76f9-a1a9-4b52-95c4-648793a021b0 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:21.715857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa76f9-a1a9-4b52-95c4-648793a021b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:21.717423 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7619542-38c0-4611-998d-e188924baad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:21.718368 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7619542-38c0-4611-998d-e188924baad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:21.718777 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7619542-38c0-4611-998d-e188924baad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:21.719060 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7619542-38c0-4611-998d-e188924baad5 admin admin] PATH_INFO: `/v3/projects/351a17f93f4c42b09f319a49a711a9cc/users/8afe2e9d859246ca85ce3f824b2cace1/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:21.756199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7619542-38c0-4611-998d-e188924baad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=351a17f93f4c42b09f319a49a711a9cc, user_id=8afe2e9d859246ca85ce3f824b2cace1, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:21.760887 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7619542-38c0-4611-998d-e188924baad5 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:21.792781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 929/1832] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:21 2022] PUT /identity/v3/projects/351a17f93f4c42b09f319a49a711a9cc/users/8afe2e9d859246ca85ce3f824b2cace1/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:21.806159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7619542-38c0-4611-998d-e188924baad5 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:21.825517 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-d7aa76f9-a1a9-4b52-95c4-648793a021b0 None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:21.830424 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d7aa76f9-a1a9-4b52-95c4-648793a021b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:21.970520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 904/1833] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 3 12:53:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:21.971823 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa76f9-a1a9-4b52-95c4-648793a021b0 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:22.028319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7619542-38c0-4611-998d-e188924baad5 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:22.040352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98cf0e-adc4-4d8e-9ce8-532c0ff2dd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:22.041761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98cf0e-adc4-4d8e-9ce8-532c0ff2dd6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:22.042061 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98cf0e-adc4-4d8e-9ce8-532c0ff2dd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:22.042342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98cf0e-adc4-4d8e-9ce8-532c0ff2dd6c admin admin] PATH_INFO: `/v3/projects/f97f3d22e6c84afe96c12a516901d5d7/users/86d221b3b09b4a51b99b3c9eb7c92ef5/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:22.084597 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98cf0e-adc4-4d8e-9ce8-532c0ff2dd6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97f3d22e6c84afe96c12a516901d5d7, user_id=86d221b3b09b4a51b99b3c9eb7c92ef5, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:22.090605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98cf0e-adc4-4d8e-9ce8-532c0ff2dd6c admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:22.126813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 930/1834] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:21 2022] PUT /identity/v3/projects/f97f3d22e6c84afe96c12a516901d5d7/users/86d221b3b09b4a51b99b3c9eb7c92ef5/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:22.126813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5792ce-a447-4c2d-882b-a9ab9813a30b None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:22.126813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5792ce-a447-4c2d-882b-a9ab9813a30b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:22.126813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5792ce-a447-4c2d-882b-a9ab9813a30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:22.146097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa76f9-a1a9-4b52-95c4-648793a021b0 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:22.175492 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-ad5792ce-a447-4c2d-882b-a9ab9813a30b None None] MFA Rules not processed for user `13128464bb2547f5838a787d022e6c5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:22.177464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ad5792ce-a447-4c2d-882b-a9ab9813a30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:22.239924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:22.302275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa7563-c50c-41dd-8b6d-9098404b2d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:22.305652 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa7563-c50c-41dd-8b6d-9098404b2d22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:22.305999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa7563-c50c-41dd-8b6d-9098404b2d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:22.306282 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa7563-c50c-41dd-8b6d-9098404b2d22 admin admin] PATH_INFO: `/v3/projects/0c4203b821c142859d098893f015fcd7/users/294c86c679d94f56979ec6961c69c235/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:22.341258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 931/1835] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:22.342805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5a88b-155f-4758-8db5-31d420f88160 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:22.343092 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5a88b-155f-4758-8db5-31d420f88160 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:22.343367 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5a88b-155f-4758-8db5-31d420f88160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:22.345380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa7563-c50c-41dd-8b6d-9098404b2d22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c4203b821c142859d098893f015fcd7, user_id=294c86c679d94f56979ec6961c69c235, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:22.348744 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa7563-c50c-41dd-8b6d-9098404b2d22 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:22.377454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 905/1836] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:21 2022] PUT /identity/v3/projects/0c4203b821c142859d098893f015fcd7/users/294c86c679d94f56979ec6961c69c235/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:22.378734 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa7563-c50c-41dd-8b6d-9098404b2d22 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:22.400980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-50d5a88b-155f-4758-8db5-31d420f88160 None None] MFA Rules not processed for user `8afe2e9d859246ca85ce3f824b2cace1`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:22.402286 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-50d5a88b-155f-4758-8db5-31d420f88160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:22.490468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:22.538706 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 932/1837] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:22.539770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5a88b-155f-4758-8db5-31d420f88160 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:22.542939 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-50d5a88b-155f-4758-8db5-31d420f88160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:22.571021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa7563-c50c-41dd-8b6d-9098404b2d22 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:22.589212 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c8a9b-db94-4f3c-8393-12c445cbeed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:22.589926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c8a9b-db94-4f3c-8393-12c445cbeed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:22.593160 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c8a9b-db94-4f3c-8393-12c445cbeed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:22.593648 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c8a9b-db94-4f3c-8393-12c445cbeed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:22.594403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c8a9b-db94-4f3c-8393-12c445cbeed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:22.596095 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c8a9b-db94-4f3c-8393-12c445cbeed1 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:22.635672 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 906/1838] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:22 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:22.636945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44353807-d7ac-41b0-94a1-54f2e88e5dab None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:22.637539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44353807-d7ac-41b0-94a1-54f2e88e5dab None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:22.638865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44353807-d7ac-41b0-94a1-54f2e88e5dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:22.699153 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-44353807-d7ac-41b0-94a1-54f2e88e5dab None None] MFA Rules not processed for user `294c86c679d94f56979ec6961c69c235`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:22.701267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-44353807-d7ac-41b0-94a1-54f2e88e5dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:22.769772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5a88b-155f-4758-8db5-31d420f88160 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:22.786692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272a950-4059-478c-b107-3e74e09f8d25 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:22.787803 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272a950-4059-478c-b107-3e74e09f8d25 service swift] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:22.787803 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272a950-4059-478c-b107-3e74e09f8d25 service swift] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:22.791503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272a950-4059-478c-b107-3e74e09f8d25 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:22.821996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 907/1839] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:22.823105 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44353807-d7ac-41b0-94a1-54f2e88e5dab None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:22.838302 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44353807-d7ac-41b0-94a1-54f2e88e5dab None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:22.848040 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6483dd-e81f-4373-9b77-2f24ab84b7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:22.848738 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6483dd-e81f-4373-9b77-2f24ab84b7b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:22.849103 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6483dd-e81f-4373-9b77-2f24ab84b7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:22.849387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6483dd-e81f-4373-9b77-2f24ab84b7b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:22.850161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6483dd-e81f-4373-9b77-2f24ab84b7b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:22.853282 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6483dd-e81f-4373-9b77-2f24ab84b7b4 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:22.924064 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 908/1840] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:22 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:22.929151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6483dd-e81f-4373-9b77-2f24ab84b7b4 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:22.966154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272a950-4059-478c-b107-3e74e09f8d25 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:22.969115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272a950-4059-478c-b107-3e74e09f8d25 service swift] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:22.982074 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 933/1841] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:53:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 12:53:22.983154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272a950-4059-478c-b107-3e74e09f8d25 service swift] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:22.986904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:23.011743 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272a950-4059-478c-b107-3e74e09f8d25 service swift] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:23.035071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6483dd-e81f-4373-9b77-2f24ab84b7b4 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:23.046976 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366e4f7-c0b7-49d8-870f-0019052ec948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:23.047859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366e4f7-c0b7-49d8-870f-0019052ec948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:23.048617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366e4f7-c0b7-49d8-870f-0019052ec948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:23.048617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366e4f7-c0b7-49d8-870f-0019052ec948 admin admin] PATH_INFO: `/v3/projects/f97f3d22e6c84afe96c12a516901d5d7/users/86d221b3b09b4a51b99b3c9eb7c92ef5/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:23.052035 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366e4f7-c0b7-49d8-870f-0019052ec948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97f3d22e6c84afe96c12a516901d5d7, user_id=86d221b3b09b4a51b99b3c9eb7c92ef5, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:23.058254 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366e4f7-c0b7-49d8-870f-0019052ec948 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:23.082991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 909/1842] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:22 2022] PUT /identity/v3/projects/f97f3d22e6c84afe96c12a516901d5d7/users/86d221b3b09b4a51b99b3c9eb7c92ef5/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:23.084146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366e4f7-c0b7-49d8-870f-0019052ec948 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:23.207574 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e2fd7-8f61-4226-8f46-f3ceab9329fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:23.208676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e2fd7-8f61-4226-8f46-f3ceab9329fb admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:23.209160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e2fd7-8f61-4226-8f46-f3ceab9329fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:23.209669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e2fd7-8f61-4226-8f46-f3ceab9329fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:23.210598 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e2fd7-8f61-4226-8f46-f3ceab9329fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:23.212803 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35e2fd7-8f61-4226-8f46-f3ceab9329fb admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:23.242265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:23.269054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366e4f7-c0b7-49d8-870f-0019052ec948 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:23.292174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20537a6-8b34-4720-9ebc-14fde2e77dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:23.292945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20537a6-8b34-4720-9ebc-14fde2e77dbe admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:23.293279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20537a6-8b34-4720-9ebc-14fde2e77dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:23.293618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20537a6-8b34-4720-9ebc-14fde2e77dbe admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:23.294322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20537a6-8b34-4720-9ebc-14fde2e77dbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:23.296411 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20537a6-8b34-4720-9ebc-14fde2e77dbe admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:23.327297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 934/1843] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:22 2022] POST /identity/v3/projects => generated 379 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:23.329578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e2fd7-8f61-4226-8f46-f3ceab9329fb admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:23.337652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a35e2fd7-8f61-4226-8f46-f3ceab9329fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:23.368320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 910/1844] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:23 2022] POST /identity/v3/users => generated 296 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:23.369734 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f14c12-1ecb-44c2-a877-9409c5501c39 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:23.370025 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f14c12-1ecb-44c2-a877-9409c5501c39 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:23.370306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f14c12-1ecb-44c2-a877-9409c5501c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:23.436285 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-31f14c12-1ecb-44c2-a877-9409c5501c39 None None] MFA Rules not processed for user `86d221b3b09b4a51b99b3c9eb7c92ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:23.437567 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-31f14c12-1ecb-44c2-a877-9409c5501c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:23.582593 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e2fd7-8f61-4226-8f46-f3ceab9329fb admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:23.590288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b60ed0-2c58-4a81-b5d5-453ba0b27035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:23.591451 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b60ed0-2c58-4a81-b5d5-453ba0b27035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:23.592101 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b60ed0-2c58-4a81-b5d5-453ba0b27035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:23.592514 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b60ed0-2c58-4a81-b5d5-453ba0b27035 admin admin] PATH_INFO: `/v3/projects/9c9a46618ac346c9bd563dff92235fcd` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:23.593999 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b60ed0-2c58-4a81-b5d5-453ba0b27035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c9a46618ac346c9bd563dff92235fcd)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:23.600969 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b60ed0-2c58-4a81-b5d5-453ba0b27035 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:23.698032 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 911/1845] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:23.700029 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f14c12-1ecb-44c2-a877-9409c5501c39 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:23.750039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f14c12-1ecb-44c2-a877-9409c5501c39 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:23.756414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-25b60ed0-2c58-4a81-b5d5-453ba0b27035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c9a46618ac346c9bd563dff92235fcd', 'request_id': 'req-25b60ed0-2c58-4a81-b5d5-453ba0b27035'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:23.785944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-25b60ed0-2c58-4a81-b5d5-453ba0b27035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c9a46618ac346c9bd563dff92235fcd', 'request_id': 'req-25b60ed0-2c58-4a81-b5d5-453ba0b27035'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:23.817111 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-25b60ed0-2c58-4a81-b5d5-453ba0b27035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c9a46618ac346c9bd563dff92235fcd', 'request_id': 'req-25b60ed0-2c58-4a81-b5d5-453ba0b27035'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:23.853106 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-25b60ed0-2c58-4a81-b5d5-453ba0b27035 admin admin] The token cache is being invalidate because project 9c9a46618ac346c9bd563dff92235fcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:23.854352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-25b60ed0-2c58-4a81-b5d5-453ba0b27035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:23.865190 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-25b60ed0-2c58-4a81-b5d5-453ba0b27035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:23.872571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 935/1846] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:23 2022] DELETE /identity/v3/projects/9c9a46618ac346c9bd563dff92235fcd => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:23.872571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b60ed0-2c58-4a81-b5d5-453ba0b27035 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:23.882639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:23.887260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-25b60ed0-2c58-4a81-b5d5-453ba0b27035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:23.976012 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb8389-c70a-4396-8a85-60d06220b4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:23.976662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb8389-c70a-4396-8a85-60d06220b4ad admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:23.976939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb8389-c70a-4396-8a85-60d06220b4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:23.977239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb8389-c70a-4396-8a85-60d06220b4ad admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:23.978105 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb8389-c70a-4396-8a85-60d06220b4ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:23.980053 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb8389-c70a-4396-8a85-60d06220b4ad admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:24.025783 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 912/1847] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:23 2022] POST /identity/v3/users => generated 296 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:24.026761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb8389-c70a-4396-8a85-60d06220b4ad admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:24.038664 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f2bb8389-c70a-4396-8a85-60d06220b4ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:24.116494 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b60ed0-2c58-4a81-b5d5-453ba0b27035 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:24.131258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeab932-3d6c-4f76-a3a1-bafe9284533a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:24.132067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeab932-3d6c-4f76-a3a1-bafe9284533a admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:24.133200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeab932-3d6c-4f76-a3a1-bafe9284533a admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:24.135475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeab932-3d6c-4f76-a3a1-bafe9284533a admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:24.136285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeab932-3d6c-4f76-a3a1-bafe9284533a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:24.151972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeab932-3d6c-4f76-a3a1-bafe9284533a admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:24.187788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 936/1848] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:24.194140 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeab932-3d6c-4f76-a3a1-bafe9284533a admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:24.197575 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb8389-c70a-4396-8a85-60d06220b4ad admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:24.199343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-9aeab932-3d6c-4f76-a3a1-bafe9284533a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:24.210756 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c31481-3010-4d8a-afcd-151c1b0ec2d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:24.211796 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c31481-3010-4d8a-afcd-151c1b0ec2d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:24.212275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c31481-3010-4d8a-afcd-151c1b0ec2d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:24.212749 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c31481-3010-4d8a-afcd-151c1b0ec2d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:24.219208 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f2c31481-3010-4d8a-afcd-151c1b0ec2d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:24.236003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeab932-3d6c-4f76-a3a1-bafe9284533a admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:24.247821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181fe4e-4e1e-45ed-80ee-8c38efdd1226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:24.248115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181fe4e-4e1e-45ed-80ee-8c38efdd1226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:24.248400 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181fe4e-4e1e-45ed-80ee-8c38efdd1226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:24.248723 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181fe4e-4e1e-45ed-80ee-8c38efdd1226 admin admin] PATH_INFO: `/v3/users/c42b23eea5c24109a4b945761c1bf8c8` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:24.253351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181fe4e-4e1e-45ed-80ee-8c38efdd1226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c42b23eea5c24109a4b945761c1bf8c8)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:24.260002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c181fe4e-4e1e-45ed-80ee-8c38efdd1226 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:24.293086 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:24.374705 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c31481-3010-4d8a-afcd-151c1b0ec2d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:24.376307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c31481-3010-4d8a-afcd-151c1b0ec2d2 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:24.376552 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c181fe4e-4e1e-45ed-80ee-8c38efdd1226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c42b23eea5c24109a4b945761c1bf8c8'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:24.393349 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c181fe4e-4e1e-45ed-80ee-8c38efdd1226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c42b23eea5c24109a4b945761c1bf8c8'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:24.412947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c181fe4e-4e1e-45ed-80ee-8c38efdd1226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c42b23eea5c24109a4b945761c1bf8c8'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:24.413909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c181fe4e-4e1e-45ed-80ee-8c38efdd1226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c42b23eea5c24109a4b945761c1bf8c8'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:24.414840 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c181fe4e-4e1e-45ed-80ee-8c38efdd1226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c42b23eea5c24109a4b945761c1bf8c8'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:24.458446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 913/1849] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:24.458446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c31481-3010-4d8a-afcd-151c1b0ec2d2 service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:24.461181 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f2c31481-3010-4d8a-afcd-151c1b0ec2d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:24.477842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 937/1850] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:24 2022] DELETE /identity/v3/users/c42b23eea5c24109a4b945761c1bf8c8 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:24.477842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181fe4e-4e1e-45ed-80ee-8c38efdd1226 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:24.481285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c181fe4e-4e1e-45ed-80ee-8c38efdd1226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:24.518554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c31481-3010-4d8a-afcd-151c1b0ec2d2 service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:24.660756 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181fe4e-4e1e-45ed-80ee-8c38efdd1226 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:24.666392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822149b-22b2-41c1-9d70-e62c05dc8294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:24.667089 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822149b-22b2-41c1-9d70-e62c05dc8294 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:24.673763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5a766-fdcb-451d-9e4b-e196e3a3efd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:24.673883 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5a766-fdcb-451d-9e4b-e196e3a3efd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:24.674106 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5a766-fdcb-451d-9e4b-e196e3a3efd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:24.676930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db5a766-fdcb-451d-9e4b-e196e3a3efd2 admin admin] PATH_INFO: `/v3/projects/5ba0ff4745be4cd78e6ba900c724b624/users/3bd7338ec96547139b07b3affe60f507/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:24.692996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822149b-22b2-41c1-9d70-e62c05dc8294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:24.701398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822149b-22b2-41c1-9d70-e62c05dc8294 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:24.702212 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822149b-22b2-41c1-9d70-e62c05dc8294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:24.715087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822149b-22b2-41c1-9d70-e62c05dc8294 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:24.728304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5a766-fdcb-451d-9e4b-e196e3a3efd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba0ff4745be4cd78e6ba900c724b624, user_id=3bd7338ec96547139b07b3affe60f507, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:24.732875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db5a766-fdcb-451d-9e4b-e196e3a3efd2 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:24.768826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 938/1851] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:24 2022] PUT /identity/v3/projects/5ba0ff4745be4cd78e6ba900c724b624/users/3bd7338ec96547139b07b3affe60f507/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:24.769592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5a766-fdcb-451d-9e4b-e196e3a3efd2 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:24.780233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 914/1852] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:24.785071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822149b-22b2-41c1-9d70-e62c05dc8294 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:24.792697 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-3822149b-22b2-41c1-9d70-e62c05dc8294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:24.856425 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822149b-22b2-41c1-9d70-e62c05dc8294 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:24.885815 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:24.950818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db5a766-fdcb-451d-9e4b-e196e3a3efd2 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:24.963213 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789cdac-f695-44b5-8799-4661851ccc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:24.963855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789cdac-f695-44b5-8799-4661851ccc01 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:24.964136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789cdac-f695-44b5-8799-4661851ccc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:24.964406 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789cdac-f695-44b5-8799-4661851ccc01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:24.965107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789cdac-f695-44b5-8799-4661851ccc01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:24.970355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789cdac-f695-44b5-8799-4661851ccc01 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:24.991814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 939/1853] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:24.992801 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789cdac-f695-44b5-8799-4661851ccc01 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:25.017330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789cdac-f695-44b5-8799-4661851ccc01 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:25.026806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bed3e-ebfd-4bd9-8cc2-613615603fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:25.027771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bed3e-ebfd-4bd9-8cc2-613615603fac admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:25.028644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bed3e-ebfd-4bd9-8cc2-613615603fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:25.028644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bed3e-ebfd-4bd9-8cc2-613615603fac admin admin] PATH_INFO: `/v3/projects/5ba0ff4745be4cd78e6ba900c724b624/users/3bd7338ec96547139b07b3affe60f507/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:25.033368 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bed3e-ebfd-4bd9-8cc2-613615603fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba0ff4745be4cd78e6ba900c724b624, user_id=3bd7338ec96547139b07b3affe60f507, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:25.038968 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bed3e-ebfd-4bd9-8cc2-613615603fac admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:25.045344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:25.048280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c56044-5364-421d-99d9-6f7b0aaef950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:25.049905 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c56044-5364-421d-99d9-6f7b0aaef950 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:25.050509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c56044-5364-421d-99d9-6f7b0aaef950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:25.051839 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c56044-5364-421d-99d9-6f7b0aaef950 admin admin] PATH_INFO: `/v3/projects/b01d3d1488944e4a9814b91cc1785dfd/users/aa2b01b19de74c4bba4fc76b0d5e0549/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:25.067693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 940/1854] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:24 2022] PUT /identity/v3/projects/5ba0ff4745be4cd78e6ba900c724b624/users/3bd7338ec96547139b07b3affe60f507/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:25.085482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c56044-5364-421d-99d9-6f7b0aaef950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01d3d1488944e4a9814b91cc1785dfd, user_id=aa2b01b19de74c4bba4fc76b0d5e0549, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:25.091996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c56044-5364-421d-99d9-6f7b0aaef950 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:25.097584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82077c-127e-4e72-83fc-2b1836abdcd1 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:25.097945 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82077c-127e-4e72-83fc-2b1836abdcd1 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:25.098299 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82077c-127e-4e72-83fc-2b1836abdcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:25.136149 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 915/1855] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:24 2022] PUT /identity/v3/projects/b01d3d1488944e4a9814b91cc1785dfd/users/aa2b01b19de74c4bba4fc76b0d5e0549/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:25.140120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c56044-5364-421d-99d9-6f7b0aaef950 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:25.188290 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-8a82077c-127e-4e72-83fc-2b1836abdcd1 None None] MFA Rules not processed for user `3bd7338ec96547139b07b3affe60f507`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:25.198791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8a82077c-127e-4e72-83fc-2b1836abdcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:25.392279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c56044-5364-421d-99d9-6f7b0aaef950 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:25.428315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf859f-56fe-4702-9f05-2db1fe9cc6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:25.428315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf859f-56fe-4702-9f05-2db1fe9cc6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:25.428315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf859f-56fe-4702-9f05-2db1fe9cc6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:25.428315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf859f-56fe-4702-9f05-2db1fe9cc6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:25.428656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf859f-56fe-4702-9f05-2db1fe9cc6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:25.428656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf859f-56fe-4702-9f05-2db1fe9cc6d5 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:25.466614 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 916/1856] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:25 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:25.468117 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf859f-56fe-4702-9f05-2db1fe9cc6d5 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:25.483647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf859f-56fe-4702-9f05-2db1fe9cc6d5 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:25.493684 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc8303-75fc-4ab3-a5c0-9037f038ee2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:25.495125 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc8303-75fc-4ab3-a5c0-9037f038ee2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:25.495521 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc8303-75fc-4ab3-a5c0-9037f038ee2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:25.495942 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc8303-75fc-4ab3-a5c0-9037f038ee2b admin admin] PATH_INFO: `/v3/projects/b01d3d1488944e4a9814b91cc1785dfd/users/aa2b01b19de74c4bba4fc76b0d5e0549/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:25.499414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc8303-75fc-4ab3-a5c0-9037f038ee2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01d3d1488944e4a9814b91cc1785dfd, user_id=aa2b01b19de74c4bba4fc76b0d5e0549, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:25.503714 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc8303-75fc-4ab3-a5c0-9037f038ee2b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:25.522555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 917/1857] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:25 2022] PUT /identity/v3/projects/b01d3d1488944e4a9814b91cc1785dfd/users/aa2b01b19de74c4bba4fc76b0d5e0549/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:25.531719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90ff54-c0a4-4346-86bf-5d0fc00617d4 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:25.532519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90ff54-c0a4-4346-86bf-5d0fc00617d4 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:25.532901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90ff54-c0a4-4346-86bf-5d0fc00617d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:25.569170 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 941/1858] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:53:25.580276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82077c-127e-4e72-83fc-2b1836abdcd1 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:25.581757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8a82077c-127e-4e72-83fc-2b1836abdcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:25.660141 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-5b90ff54-c0a4-4346-86bf-5d0fc00617d4 None None] MFA Rules not processed for user `aa2b01b19de74c4bba4fc76b0d5e0549`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:25.668028 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-5b90ff54-c0a4-4346-86bf-5d0fc00617d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:25.812962 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82077c-127e-4e72-83fc-2b1836abdcd1 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:25.907585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 918/1859] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:25 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:25.923110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90ff54-c0a4-4346-86bf-5d0fc00617d4 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:25.934003 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-5b90ff54-c0a4-4346-86bf-5d0fc00617d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:25.967388 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:26.034977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b5ec6-1e3f-471f-b560-bccaa183cba5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:26.035659 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b5ec6-1e3f-471f-b560-bccaa183cba5 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:26.035944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b5ec6-1e3f-471f-b560-bccaa183cba5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:26.036223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b5ec6-1e3f-471f-b560-bccaa183cba5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:26.174148 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:26.187869 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90ff54-c0a4-4346-86bf-5d0fc00617d4 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:26.194332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b5ec6-1e3f-471f-b560-bccaa183cba5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:26.196295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b5ec6-1e3f-471f-b560-bccaa183cba5 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:26.216711 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805e57d-7424-4100-8229-2af49c514cd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:26.222096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 942/1860] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:26.230222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805e57d-7424-4100-8229-2af49c514cd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:26.230600 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805e57d-7424-4100-8229-2af49c514cd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:26.233254 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805e57d-7424-4100-8229-2af49c514cd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:26.268776 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805e57d-7424-4100-8229-2af49c514cd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:26.275142 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805e57d-7424-4100-8229-2af49c514cd9 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:26.317195 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 919/1861] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:25 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:26.495513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b5ec6-1e3f-471f-b560-bccaa183cba5 service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:26.515325 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b5ec6-1e3f-471f-b560-bccaa183cba5 service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:26.531179 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac8c4f-ffd6-4b20-9473-f041049d755f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:26.533400 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac8c4f-ffd6-4b20-9473-f041049d755f service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:26.533400 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac8c4f-ffd6-4b20-9473-f041049d755f service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:26.533545 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac8c4f-ffd6-4b20-9473-f041049d755f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:26.535935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805e57d-7424-4100-8229-2af49c514cd9 service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:26.542022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8cac8c4f-ffd6-4b20-9473-f041049d755f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:26.565674 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805e57d-7424-4100-8229-2af49c514cd9 service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:26.591811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8486057-59d4-4a5a-a473-6f0d1a6694cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:26.596082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8486057-59d4-4a5a-a473-6f0d1a6694cc service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:26.597097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8486057-59d4-4a5a-a473-6f0d1a6694cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:26.599129 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8486057-59d4-4a5a-a473-6f0d1a6694cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:26.628116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a8486057-59d4-4a5a-a473-6f0d1a6694cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:26.753767 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac8c4f-ffd6-4b20-9473-f041049d755f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:26.758617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cac8c4f-ffd6-4b20-9473-f041049d755f service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:26.810581 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 943/1862] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:26.939095 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8486057-59d4-4a5a-a473-6f0d1a6694cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:26.941454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8486057-59d4-4a5a-a473-6f0d1a6694cc service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:27.026958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 920/1863] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:27.314817 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac8c4f-ffd6-4b20-9473-f041049d755f service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:27.317456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8cac8c4f-ffd6-4b20-9473-f041049d755f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:27.371891 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520f7a8-2d82-4515-9a5d-2788999ecb05 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:27.372355 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520f7a8-2d82-4515-9a5d-2788999ecb05 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:27.372787 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520f7a8-2d82-4515-9a5d-2788999ecb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:27.451694 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-8520f7a8-2d82-4515-9a5d-2788999ecb05 None None] MFA Rules not processed for user `52ff458fd7f74b99a745edbcf13a114b`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:27.457605 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8520f7a8-2d82-4515-9a5d-2788999ecb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:27.538809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac8c4f-ffd6-4b20-9473-f041049d755f service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:27.556769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1fcfe6-27e1-480e-baf1-3732ebef8bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:27.561502 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fcfe6-27e1-480e-baf1-3732ebef8bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:27.561763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fcfe6-27e1-480e-baf1-3732ebef8bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:27.563731 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1fcfe6-27e1-480e-baf1-3732ebef8bce admin admin] PATH_INFO: `/v3/projects/ab36185a7ea44ae5b01e14927a875797` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:27.567771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fcfe6-27e1-480e-baf1-3732ebef8bce admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab36185a7ea44ae5b01e14927a875797)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:27.573518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1fcfe6-27e1-480e-baf1-3732ebef8bce admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:27.692297 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 921/1864] 10.222.0.77 () {58 vars in 1015 bytes} [Wed Aug 3 12:53:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:53:27.720776 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fe1fcfe6-27e1-480e-baf1-3732ebef8bce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab36185a7ea44ae5b01e14927a875797', 'request_id': 'req-fe1fcfe6-27e1-480e-baf1-3732ebef8bce'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:27.742035 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fe1fcfe6-27e1-480e-baf1-3732ebef8bce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab36185a7ea44ae5b01e14927a875797', 'request_id': 'req-fe1fcfe6-27e1-480e-baf1-3732ebef8bce'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:27.760846 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fe1fcfe6-27e1-480e-baf1-3732ebef8bce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab36185a7ea44ae5b01e14927a875797', 'request_id': 'req-fe1fcfe6-27e1-480e-baf1-3732ebef8bce'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:27.794383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fe1fcfe6-27e1-480e-baf1-3732ebef8bce admin admin] The token cache is being invalidate because project ab36185a7ea44ae5b01e14927a875797 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:27.795615 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fe1fcfe6-27e1-480e-baf1-3732ebef8bce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:27.798001 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fe1fcfe6-27e1-480e-baf1-3732ebef8bce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:27.802803 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 944/1865] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:27 2022] DELETE /identity/v3/projects/ab36185a7ea44ae5b01e14927a875797 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:27.807641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a68684-e4b5-48b5-b040-f83a1687915b None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:27.807939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a68684-e4b5-48b5-b040-f83a1687915b None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:27.808217 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a68684-e4b5-48b5-b040-f83a1687915b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:27.873416 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-30a68684-e4b5-48b5-b040-f83a1687915b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:27.877114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-30a68684-e4b5-48b5-b040-f83a1687915b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:28.050637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:28.068222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 922/1866] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:28.076032 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64e25c-c5a7-461d-a7d4-34f82feee6e9 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:28.076689 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64e25c-c5a7-461d-a7d4-34f82feee6e9 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:28.077119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64e25c-c5a7-461d-a7d4-34f82feee6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:28.112834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-4c64e25c-c5a7-461d-a7d4-34f82feee6e9 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:28.114760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-4c64e25c-c5a7-461d-a7d4-34f82feee6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:28.123983 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 945/1867] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:28.133536 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a68684-e4b5-48b5-b040-f83a1687915b None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:28.167362 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a68684-e4b5-48b5-b040-f83a1687915b None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:28.180696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a53261-d919-49e5-ba5e-a6fd586cb9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:28.181331 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a53261-d919-49e5-ba5e-a6fd586cb9bd admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:28.181610 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a53261-d919-49e5-ba5e-a6fd586cb9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:28.181881 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a53261-d919-49e5-ba5e-a6fd586cb9bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:28.183039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a53261-d919-49e5-ba5e-a6fd586cb9bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:53:28.183039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a53261-d919-49e5-ba5e-a6fd586cb9bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:28.184641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a53261-d919-49e5-ba5e-a6fd586cb9bd admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:28.213463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 923/1868] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:53:28.214710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64e25c-c5a7-461d-a7d4-34f82feee6e9 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:28.235051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64e25c-c5a7-461d-a7d4-34f82feee6e9 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:28.239904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c988e1-de02-4533-bd1f-1df3778ea565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:28.240523 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c988e1-de02-4533-bd1f-1df3778ea565 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:28.240804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c988e1-de02-4533-bd1f-1df3778ea565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:28.241076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c988e1-de02-4533-bd1f-1df3778ea565 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:28.241749 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c988e1-de02-4533-bd1f-1df3778ea565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:28.243673 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c988e1-de02-4533-bd1f-1df3778ea565 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:28.248484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:28.274862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 946/1869] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:28 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:53:28.279025 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a53261-d919-49e5-ba5e-a6fd586cb9bd admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:28.289115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a53261-d919-49e5-ba5e-a6fd586cb9bd admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:28.349100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bfca9-db17-4599-a52f-bd90e21e65b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:28.349722 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bfca9-db17-4599-a52f-bd90e21e65b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:28.350007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bfca9-db17-4599-a52f-bd90e21e65b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:28.350284 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9bfca9-db17-4599-a52f-bd90e21e65b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:28.351030 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9bfca9-db17-4599-a52f-bd90e21e65b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:28.352979 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9bfca9-db17-4599-a52f-bd90e21e65b3 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:28.374103 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 924/1870] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:28 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:53:28.378956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c988e1-de02-4533-bd1f-1df3778ea565 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:28.389041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c988e1-de02-4533-bd1f-1df3778ea565 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:28.394185 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263f6dd-701c-493f-8b9b-d962932ec6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:28.394832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263f6dd-701c-493f-8b9b-d962932ec6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:28.395110 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263f6dd-701c-493f-8b9b-d962932ec6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:28.395379 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263f6dd-701c-493f-8b9b-d962932ec6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:28.396082 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263f6dd-701c-493f-8b9b-d962932ec6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:28.397678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263f6dd-701c-493f-8b9b-d962932ec6e9 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:28.407582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 947/1871] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:28.411559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bfca9-db17-4599-a52f-bd90e21e65b3 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:28.421622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9bfca9-db17-4599-a52f-bd90e21e65b3 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:28.427902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca02c89-b70d-4733-82a0-590147484553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:28.428516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca02c89-b70d-4733-82a0-590147484553 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:28.428800 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca02c89-b70d-4733-82a0-590147484553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:28.429075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca02c89-b70d-4733-82a0-590147484553 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:28.429807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca02c89-b70d-4733-82a0-590147484553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:28.431426 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca02c89-b70d-4733-82a0-590147484553 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:28.446089 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 925/1872] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:28.448506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263f6dd-701c-493f-8b9b-d962932ec6e9 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:28.459111 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263f6dd-701c-493f-8b9b-d962932ec6e9 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:28.463693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a8ddc-6ffa-4b29-8bab-91c309a98f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:28.464361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a8ddc-6ffa-4b29-8bab-91c309a98f8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:28.464634 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a8ddc-6ffa-4b29-8bab-91c309a98f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:28.464905 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397a8ddc-6ffa-4b29-8bab-91c309a98f8b admin admin] PATH_INFO: `/v3/projects/bbd88bc81a034005bc3f186772f01383/users/3716da815a1240119e08686e6ac77875/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:28.479411 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a8ddc-6ffa-4b29-8bab-91c309a98f8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd88bc81a034005bc3f186772f01383, user_id=3716da815a1240119e08686e6ac77875, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:28.482075 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397a8ddc-6ffa-4b29-8bab-91c309a98f8b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:28.496521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 948/1873] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:28 2022] PUT /identity/v3/projects/bbd88bc81a034005bc3f186772f01383/users/3716da815a1240119e08686e6ac77875/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:28.507491 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17877635-fe44-4b3d-b29b-032716e5e415 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:28.507775 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17877635-fe44-4b3d-b29b-032716e5e415 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:28.508051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17877635-fe44-4b3d-b29b-032716e5e415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:28.535198 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-17877635-fe44-4b3d-b29b-032716e5e415 None None] MFA Rules not processed for user `3716da815a1240119e08686e6ac77875`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:28.536403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-17877635-fe44-4b3d-b29b-032716e5e415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:28.606414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 926/1874] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:53:28.611061 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a8ddc-6ffa-4b29-8bab-91c309a98f8b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:28.688200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397a8ddc-6ffa-4b29-8bab-91c309a98f8b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:28.694123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff19e82e-eb77-4a61-844d-0f5db6de3405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:28.694786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19e82e-eb77-4a61-844d-0f5db6de3405 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:28.695342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19e82e-eb77-4a61-844d-0f5db6de3405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:28.695342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19e82e-eb77-4a61-844d-0f5db6de3405 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:28.696075 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff19e82e-eb77-4a61-844d-0f5db6de3405 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:28.698260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff19e82e-eb77-4a61-844d-0f5db6de3405 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:28.728291 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 949/1875] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:28 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:28.732863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17877635-fe44-4b3d-b29b-032716e5e415 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:28.747384 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17877635-fe44-4b3d-b29b-032716e5e415 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:28.819336 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:28.833961 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59087263-c743-4993-9998-661c2900ebd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:28.835277 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59087263-c743-4993-9998-661c2900ebd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:28.835564 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59087263-c743-4993-9998-661c2900ebd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:28.835838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59087263-c743-4993-9998-661c2900ebd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:28.836503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59087263-c743-4993-9998-661c2900ebd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:28.838392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59087263-c743-4993-9998-661c2900ebd3 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:28.863042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 927/1876] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:28 2022] POST /identity/v3/users => generated 297 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:28.865045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff19e82e-eb77-4a61-844d-0f5db6de3405 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:28.875361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff19e82e-eb77-4a61-844d-0f5db6de3405 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:28.880182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf767b-ecd7-4616-8fb4-b3cdbfef9561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:28.880633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf767b-ecd7-4616-8fb4-b3cdbfef9561 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:28.880913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf767b-ecd7-4616-8fb4-b3cdbfef9561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:28.881188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccf767b-ecd7-4616-8fb4-b3cdbfef9561 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:28.881885 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf767b-ecd7-4616-8fb4-b3cdbfef9561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:28.883499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccf767b-ecd7-4616-8fb4-b3cdbfef9561 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:28.894126 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 950/1877] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:28.898375 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59087263-c743-4993-9998-661c2900ebd3 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:28.908375 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59087263-c743-4993-9998-661c2900ebd3 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:28.913424 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ce482-6896-4336-a5f2-ecc61e6655a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:28.914075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ce482-6896-4336-a5f2-ecc61e6655a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:28.914363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ce482-6896-4336-a5f2-ecc61e6655a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:28.914673 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ce482-6896-4336-a5f2-ecc61e6655a5 admin admin] PATH_INFO: `/v3/projects/bf6608e0a9114e0ba169729577c54fd0/users/384a8868396c477197c79d4df4eaa48b/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:28.931042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ce482-6896-4336-a5f2-ecc61e6655a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf6608e0a9114e0ba169729577c54fd0, user_id=384a8868396c477197c79d4df4eaa48b, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:28.933621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ce482-6896-4336-a5f2-ecc61e6655a5 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:28.945772 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 928/1878] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:28 2022] PUT /identity/v3/projects/bf6608e0a9114e0ba169729577c54fd0/users/384a8868396c477197c79d4df4eaa48b/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:28.949684 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf767b-ecd7-4616-8fb4-b3cdbfef9561 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:28.959820 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccf767b-ecd7-4616-8fb4-b3cdbfef9561 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:29.016904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4cf20-1ac4-412a-b105-a89e09b14bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:29.017528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4cf20-1ac4-412a-b105-a89e09b14bdf admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:29.017809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4cf20-1ac4-412a-b105-a89e09b14bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:29.018121 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4cf20-1ac4-412a-b105-a89e09b14bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:29.018867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4cf20-1ac4-412a-b105-a89e09b14bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:29.020452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4cf20-1ac4-412a-b105-a89e09b14bdf admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:29.030537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 951/1879] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:29.035482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ce482-6896-4336-a5f2-ecc61e6655a5 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:29.046505 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ce482-6896-4336-a5f2-ecc61e6655a5 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:29.052192 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce81c2a-edd1-4bc0-8f66-41a874a175b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:29.052864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce81c2a-edd1-4bc0-8f66-41a874a175b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:29.053431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce81c2a-edd1-4bc0-8f66-41a874a175b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:29.053431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce81c2a-edd1-4bc0-8f66-41a874a175b6 admin admin] PATH_INFO: `/v3/projects/bf6608e0a9114e0ba169729577c54fd0/users/384a8868396c477197c79d4df4eaa48b/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:29.055371 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce81c2a-edd1-4bc0-8f66-41a874a175b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf6608e0a9114e0ba169729577c54fd0, user_id=384a8868396c477197c79d4df4eaa48b, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:29.058028 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce81c2a-edd1-4bc0-8f66-41a874a175b6 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:29.071097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 929/1880] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:29 2022] PUT /identity/v3/projects/bf6608e0a9114e0ba169729577c54fd0/users/384a8868396c477197c79d4df4eaa48b/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:29.082866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657d65e-5cb3-4189-911b-198fb3a05d72 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:29.083675 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657d65e-5cb3-4189-911b-198fb3a05d72 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:29.083675 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657d65e-5cb3-4189-911b-198fb3a05d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:29.118122 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-7657d65e-5cb3-4189-911b-198fb3a05d72 None None] MFA Rules not processed for user `384a8868396c477197c79d4df4eaa48b`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:29.124647 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-7657d65e-5cb3-4189-911b-198fb3a05d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:29.133640 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:29.195558 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 952/1881] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:29.206698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce81c2a-edd1-4bc0-8f66-41a874a175b6 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:29.208427 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-bce81c2a-edd1-4bc0-8f66-41a874a175b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:29.265111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce81c2a-edd1-4bc0-8f66-41a874a175b6 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:29.269230 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a882e94-4c26-4958-a9d6-442cd348c474 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:29.269846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a882e94-4c26-4958-a9d6-442cd348c474 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:29.270121 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a882e94-4c26-4958-a9d6-442cd348c474 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:29.270391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a882e94-4c26-4958-a9d6-442cd348c474 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:29.279465 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a882e94-4c26-4958-a9d6-442cd348c474 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:29.280877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a882e94-4c26-4958-a9d6-442cd348c474 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:29.290971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 930/1882] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:29.673882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657d65e-5cb3-4189-911b-198fb3a05d72 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:29.736708 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657d65e-5cb3-4189-911b-198fb3a05d72 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:29.742117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858aa4e-be1d-43f8-9da8-12fc989089fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:29.742797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858aa4e-be1d-43f8-9da8-12fc989089fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:29.743079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858aa4e-be1d-43f8-9da8-12fc989089fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:29.743352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858aa4e-be1d-43f8-9da8-12fc989089fe admin admin] PATH_INFO: `/v3/users/3716da815a1240119e08686e6ac77875` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:29.744512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858aa4e-be1d-43f8-9da8-12fc989089fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=3716da815a1240119e08686e6ac77875)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:29.746418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858aa4e-be1d-43f8-9da8-12fc989089fe admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:29.791491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1858aa4e-be1d-43f8-9da8-12fc989089fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3716da815a1240119e08686e6ac77875'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:29.798511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1858aa4e-be1d-43f8-9da8-12fc989089fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3716da815a1240119e08686e6ac77875'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:29.810761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1858aa4e-be1d-43f8-9da8-12fc989089fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3716da815a1240119e08686e6ac77875'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:29.811419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1858aa4e-be1d-43f8-9da8-12fc989089fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3716da815a1240119e08686e6ac77875'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:29.812012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1858aa4e-be1d-43f8-9da8-12fc989089fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3716da815a1240119e08686e6ac77875'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:29.824094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 953/1883] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:29 2022] DELETE /identity/v3/users/3716da815a1240119e08686e6ac77875 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:29.828622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dea9b3-c279-46ca-88bb-613be15a73c0 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:29.829138 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dea9b3-c279-46ca-88bb-613be15a73c0 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:29.829230 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dea9b3-c279-46ca-88bb-613be15a73c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:29.848790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-69dea9b3-c279-46ca-88bb-613be15a73c0 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:29.849977 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-69dea9b3-c279-46ca-88bb-613be15a73c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:29.911538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 931/1884] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:29.921063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858aa4e-be1d-43f8-9da8-12fc989089fe admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:29.922804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1858aa4e-be1d-43f8-9da8-12fc989089fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:29.951385 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:29.982045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858aa4e-be1d-43f8-9da8-12fc989089fe admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:29.985955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8639f8f-5149-42fe-b355-d937b20e0752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:29.986608 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8639f8f-5149-42fe-b355-d937b20e0752 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:29.987451 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8639f8f-5149-42fe-b355-d937b20e0752 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:29.987673 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8639f8f-5149-42fe-b355-d937b20e0752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:29.996613 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8639f8f-5149-42fe-b355-d937b20e0752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:29.998033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8639f8f-5149-42fe-b355-d937b20e0752 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:30.007983 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 954/1885] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:53:30.332235 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dea9b3-c279-46ca-88bb-613be15a73c0 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:30.333752 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-69dea9b3-c279-46ca-88bb-613be15a73c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:30.347730 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dea9b3-c279-46ca-88bb-613be15a73c0 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:30.352441 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c7b26-3e37-40f1-a68f-61335edc4349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:30.353164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c7b26-3e37-40f1-a68f-61335edc4349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:30.353515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c7b26-3e37-40f1-a68f-61335edc4349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:30.353922 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c7b26-3e37-40f1-a68f-61335edc4349 admin admin] PATH_INFO: `/v3/projects/bbd88bc81a034005bc3f186772f01383` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:30.355526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c7b26-3e37-40f1-a68f-61335edc4349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbd88bc81a034005bc3f186772f01383)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:30.357923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c7b26-3e37-40f1-a68f-61335edc4349 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:30.398679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:30.405961 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1a1c7b26-3e37-40f1-a68f-61335edc4349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbd88bc81a034005bc3f186772f01383', 'request_id': 'req-1a1c7b26-3e37-40f1-a68f-61335edc4349'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:30.414112 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1a1c7b26-3e37-40f1-a68f-61335edc4349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbd88bc81a034005bc3f186772f01383', 'request_id': 'req-1a1c7b26-3e37-40f1-a68f-61335edc4349'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:30.420328 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1a1c7b26-3e37-40f1-a68f-61335edc4349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbd88bc81a034005bc3f186772f01383', 'request_id': 'req-1a1c7b26-3e37-40f1-a68f-61335edc4349'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:30.431884 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1a1c7b26-3e37-40f1-a68f-61335edc4349 admin admin] The token cache is being invalidate because project bbd88bc81a034005bc3f186772f01383 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:30.432170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1a1c7b26-3e37-40f1-a68f-61335edc4349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:30.432798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1a1c7b26-3e37-40f1-a68f-61335edc4349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:30.434294 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 932/1886] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:30 2022] DELETE /identity/v3/projects/bbd88bc81a034005bc3f186772f01383 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:30.438747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8639f8f-5149-42fe-b355-d937b20e0752 service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:30.448704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8639f8f-5149-42fe-b355-d937b20e0752 service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:30.526428 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc845b3f-1312-4932-b50e-be32fc070b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:30.527309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc845b3f-1312-4932-b50e-be32fc070b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:30.527829 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc845b3f-1312-4932-b50e-be32fc070b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:30.528318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc845b3f-1312-4932-b50e-be32fc070b2b admin admin] PATH_INFO: `/v3/users/384a8868396c477197c79d4df4eaa48b` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:30.531186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc845b3f-1312-4932-b50e-be32fc070b2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=384a8868396c477197c79d4df4eaa48b)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:30.532602 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc845b3f-1312-4932-b50e-be32fc070b2b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:30.633729 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fc845b3f-1312-4932-b50e-be32fc070b2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '384a8868396c477197c79d4df4eaa48b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:30.651241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fc845b3f-1312-4932-b50e-be32fc070b2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '384a8868396c477197c79d4df4eaa48b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:30.681219 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fc845b3f-1312-4932-b50e-be32fc070b2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '384a8868396c477197c79d4df4eaa48b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:30.683016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fc845b3f-1312-4932-b50e-be32fc070b2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '384a8868396c477197c79d4df4eaa48b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:30.686704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fc845b3f-1312-4932-b50e-be32fc070b2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '384a8868396c477197c79d4df4eaa48b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:30.718860 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 955/1887] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:30 2022] DELETE /identity/v3/users/384a8868396c477197c79d4df4eaa48b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:31.056219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c7b26-3e37-40f1-a68f-61335edc4349 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:31.057657 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1a1c7b26-3e37-40f1-a68f-61335edc4349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:31.071039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c7b26-3e37-40f1-a68f-61335edc4349 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:31.156762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f517d-a44c-421d-961b-3497062f3af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:31.157430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f517d-a44c-421d-961b-3497062f3af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:31.157740 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f517d-a44c-421d-961b-3497062f3af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:31.158227 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f517d-a44c-421d-961b-3497062f3af5 admin admin] PATH_INFO: `/v3/projects/bf6608e0a9114e0ba169729577c54fd0` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:31.159583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f517d-a44c-421d-961b-3497062f3af5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf6608e0a9114e0ba169729577c54fd0)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:31.161683 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f517d-a44c-421d-961b-3497062f3af5 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:31.212878 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f69f517d-a44c-421d-961b-3497062f3af5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf6608e0a9114e0ba169729577c54fd0', 'request_id': 'req-f69f517d-a44c-421d-961b-3497062f3af5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:31.226101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f69f517d-a44c-421d-961b-3497062f3af5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf6608e0a9114e0ba169729577c54fd0', 'request_id': 'req-f69f517d-a44c-421d-961b-3497062f3af5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:31.232952 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f69f517d-a44c-421d-961b-3497062f3af5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf6608e0a9114e0ba169729577c54fd0', 'request_id': 'req-f69f517d-a44c-421d-961b-3497062f3af5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:31.244522 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f69f517d-a44c-421d-961b-3497062f3af5 admin admin] The token cache is being invalidate because project bf6608e0a9114e0ba169729577c54fd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:31.244854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f69f517d-a44c-421d-961b-3497062f3af5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:31.246068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f69f517d-a44c-421d-961b-3497062f3af5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:31.248180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 933/1888] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:31 2022] DELETE /identity/v3/projects/bf6608e0a9114e0ba169729577c54fd0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:53:31.253408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f16ad-74b5-49c5-bbb9-b8a5401680d0 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:31.253699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f16ad-74b5-49c5-bbb9-b8a5401680d0 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:31.253961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f16ad-74b5-49c5-bbb9-b8a5401680d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:31.275266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-385f16ad-74b5-49c5-bbb9-b8a5401680d0 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:31.276486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-385f16ad-74b5-49c5-bbb9-b8a5401680d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:31.360580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:31.416866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 956/1889] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:31.427013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f09afd-2427-46a5-b928-ff41f2493980 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:31.427497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f09afd-2427-46a5-b928-ff41f2493980 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:31.427957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f09afd-2427-46a5-b928-ff41f2493980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:31.472326 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-98f09afd-2427-46a5-b928-ff41f2493980 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:31.473884 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-98f09afd-2427-46a5-b928-ff41f2493980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:31.489709 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 934/1890] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:53:31.494222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f16ad-74b5-49c5-bbb9-b8a5401680d0 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:31.508935 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385f16ad-74b5-49c5-bbb9-b8a5401680d0 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:31.522761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447b3e7-8d6f-40e5-8733-ed16c7ecd863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:31.523842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447b3e7-8d6f-40e5-8733-ed16c7ecd863 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:31.524360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447b3e7-8d6f-40e5-8733-ed16c7ecd863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:31.524772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447b3e7-8d6f-40e5-8733-ed16c7ecd863 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:31.525608 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447b3e7-8d6f-40e5-8733-ed16c7ecd863 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:53:31.526248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447b3e7-8d6f-40e5-8733-ed16c7ecd863 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:31.534243 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447b3e7-8d6f-40e5-8733-ed16c7ecd863 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:31.551151 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 957/1891] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:53:31.561609 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f09afd-2427-46a5-b928-ff41f2493980 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:31.597451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f09afd-2427-46a5-b928-ff41f2493980 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:31.615186 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07007e9-d2d3-4bbd-9d60-fbb2bab45bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:31.623457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07007e9-d2d3-4bbd-9d60-fbb2bab45bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:31.624875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07007e9-d2d3-4bbd-9d60-fbb2bab45bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:31.625445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07007e9-d2d3-4bbd-9d60-fbb2bab45bd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:31.626927 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07007e9-d2d3-4bbd-9d60-fbb2bab45bd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:31.630111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07007e9-d2d3-4bbd-9d60-fbb2bab45bd7 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:31.701234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 935/1892] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:31 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:31.705698 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447b3e7-8d6f-40e5-8733-ed16c7ecd863 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:31.722793 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447b3e7-8d6f-40e5-8733-ed16c7ecd863 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:31.816503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07007e9-d2d3-4bbd-9d60-fbb2bab45bd7 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:31.818980 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-b07007e9-d2d3-4bbd-9d60-fbb2bab45bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:31.828781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d927a4-4a32-47f5-9544-623c3745945d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:31.829827 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d927a4-4a32-47f5-9544-623c3745945d admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:31.830222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d927a4-4a32-47f5-9544-623c3745945d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:31.830644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d927a4-4a32-47f5-9544-623c3745945d admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:31.831407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d927a4-4a32-47f5-9544-623c3745945d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:31.834465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d927a4-4a32-47f5-9544-623c3745945d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:31.870835 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 958/1893] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:31 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:31.881249 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d927a4-4a32-47f5-9544-623c3745945d admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:31.907524 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d927a4-4a32-47f5-9544-623c3745945d admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:31.920574 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:31.925589 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c200826-3f70-47aa-84a9-c29131c3afb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:31.930367 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200826-3f70-47aa-84a9-c29131c3afb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:31.935515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200826-3f70-47aa-84a9-c29131c3afb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:31.936424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c200826-3f70-47aa-84a9-c29131c3afb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:31.937968 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c200826-3f70-47aa-84a9-c29131c3afb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:31.943761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c200826-3f70-47aa-84a9-c29131c3afb1 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:31.973056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07007e9-d2d3-4bbd-9d60-fbb2bab45bd7 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:31.975991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 959/1894] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:31.977874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769045b0-0c08-474b-910d-c3b1a49ba91f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:31.978680 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769045b0-0c08-474b-910d-c3b1a49ba91f service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:31.979297 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769045b0-0c08-474b-910d-c3b1a49ba91f service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:31.979582 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769045b0-0c08-474b-910d-c3b1a49ba91f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:31.980693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c200826-3f70-47aa-84a9-c29131c3afb1 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:31.981650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-769045b0-0c08-474b-910d-c3b1a49ba91f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:32.010018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c200826-3f70-47aa-84a9-c29131c3afb1 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:32.024426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df5530-f0f3-4df1-8998-8a765e1d13e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:32.032181 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df5530-f0f3-4df1-8998-8a765e1d13e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:32.032530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df5530-f0f3-4df1-8998-8a765e1d13e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:32.033462 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df5530-f0f3-4df1-8998-8a765e1d13e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:32.033462 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df5530-f0f3-4df1-8998-8a765e1d13e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:32.037682 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df5530-f0f3-4df1-8998-8a765e1d13e0 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:32.079450 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 960/1895] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:31 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:53:32.084170 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df5530-f0f3-4df1-8998-8a765e1d13e0 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:32.104648 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df5530-f0f3-4df1-8998-8a765e1d13e0 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:32.119187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920830fb-3148-444e-a89b-a612bf1d3e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:32.119882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920830fb-3148-444e-a89b-a612bf1d3e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:32.120165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920830fb-3148-444e-a89b-a612bf1d3e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:32.120440 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920830fb-3148-444e-a89b-a612bf1d3e47 admin admin] PATH_INFO: `/v3/projects/dc4183d3440141db95f43ea9c75bb41a/users/49548220763c47b58d5ae6870e5e042f/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:32.146238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769045b0-0c08-474b-910d-c3b1a49ba91f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:32.149089 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769045b0-0c08-474b-910d-c3b1a49ba91f service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:32.158807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920830fb-3148-444e-a89b-a612bf1d3e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc4183d3440141db95f43ea9c75bb41a, user_id=49548220763c47b58d5ae6870e5e042f, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:32.161467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920830fb-3148-444e-a89b-a612bf1d3e47 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:32.164012 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 936/1896] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:53:32.188699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 961/1897] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:32 2022] PUT /identity/v3/projects/dc4183d3440141db95f43ea9c75bb41a/users/49548220763c47b58d5ae6870e5e042f/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:53:32.207436 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712b8f5-84d9-4169-b500-9432fccfaef0 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:32.207901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712b8f5-84d9-4169-b500-9432fccfaef0 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:32.208185 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9712b8f5-84d9-4169-b500-9432fccfaef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:32.257623 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-9712b8f5-84d9-4169-b500-9432fccfaef0 None None] MFA Rules not processed for user `49548220763c47b58d5ae6870e5e042f`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:32.259264 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-9712b8f5-84d9-4169-b500-9432fccfaef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:32.373423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 937/1898] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:32.387205 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920830fb-3148-444e-a89b-a612bf1d3e47 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:32.389493 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-920830fb-3148-444e-a89b-a612bf1d3e47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:32.571620 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920830fb-3148-444e-a89b-a612bf1d3e47 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:32.575908 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60343a15-98e9-43cf-b29e-4b4760773d92 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6066e48aa644a47a889e09cfe204858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:32.576679 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60343a15-98e9-43cf-b29e-4b4760773d92 service glance] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:32.577146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60343a15-98e9-43cf-b29e-4b4760773d92 service glance] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:32.577444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60343a15-98e9-43cf-b29e-4b4760773d92 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:32.588463 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60343a15-98e9-43cf-b29e-4b4760773d92 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:32.590323 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60343a15-98e9-43cf-b29e-4b4760773d92 service glance] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:32.607549 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:32.616909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 962/1899] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:33.461820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712b8f5-84d9-4169-b500-9432fccfaef0 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:33.463605 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-9712b8f5-84d9-4169-b500-9432fccfaef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:33.550707 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9712b8f5-84d9-4169-b500-9432fccfaef0 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:33.555645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b8272-7e55-455b-b4a5-ba97892f4fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:33.556288 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b8272-7e55-455b-b4a5-ba97892f4fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:33.556571 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b8272-7e55-455b-b4a5-ba97892f4fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:33.556844 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b8272-7e55-455b-b4a5-ba97892f4fef admin admin] PATH_INFO: `/v3/users/4eb0a3d0aada4db1a80f19a7267e490b` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:33.558065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b8272-7e55-455b-b4a5-ba97892f4fef admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eb0a3d0aada4db1a80f19a7267e490b)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:33.560022 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b8272-7e55-455b-b4a5-ba97892f4fef admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:33.572422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:33.606857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-159b8272-7e55-455b-b4a5-ba97892f4fef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eb0a3d0aada4db1a80f19a7267e490b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:33.624830 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-159b8272-7e55-455b-b4a5-ba97892f4fef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eb0a3d0aada4db1a80f19a7267e490b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:33.635858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-159b8272-7e55-455b-b4a5-ba97892f4fef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eb0a3d0aada4db1a80f19a7267e490b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:33.636704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-159b8272-7e55-455b-b4a5-ba97892f4fef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eb0a3d0aada4db1a80f19a7267e490b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:33.649745 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-159b8272-7e55-455b-b4a5-ba97892f4fef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eb0a3d0aada4db1a80f19a7267e490b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:33.656246 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 938/1900] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:33 2022] DELETE /identity/v3/users/4eb0a3d0aada4db1a80f19a7267e490b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:33.661169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f7427-2628-44b7-8d22-8fb09d3a6246 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:33.661654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f7427-2628-44b7-8d22-8fb09d3a6246 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:33.661740 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f7427-2628-44b7-8d22-8fb09d3a6246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:33.680745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-179f7427-2628-44b7-8d22-8fb09d3a6246 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:33.682000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-179f7427-2628-44b7-8d22-8fb09d3a6246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:33.741742 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 963/1901] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:33.751603 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b8272-7e55-455b-b4a5-ba97892f4fef admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:33.753075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-159b8272-7e55-455b-b4a5-ba97892f4fef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:33.813291 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b8272-7e55-455b-b4a5-ba97892f4fef admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:33.817002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4327d-5a68-4ec0-920d-4592a135e94c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:33.817619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4327d-5a68-4ec0-920d-4592a135e94c service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:33.817899 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4327d-5a68-4ec0-920d-4592a135e94c service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:33.818177 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4327d-5a68-4ec0-920d-4592a135e94c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:33.827256 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4327d-5a68-4ec0-920d-4592a135e94c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:33.828722 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4327d-5a68-4ec0-920d-4592a135e94c service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:33.839017 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 939/1902] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:53:34.153961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f7427-2628-44b7-8d22-8fb09d3a6246 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:34.155388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-179f7427-2628-44b7-8d22-8fb09d3a6246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:34.167727 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f7427-2628-44b7-8d22-8fb09d3a6246 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:34.172559 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c432c20-46ea-4045-8ed1-caae75f4c87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:34.173166 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c432c20-46ea-4045-8ed1-caae75f4c87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:34.173445 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c432c20-46ea-4045-8ed1-caae75f4c87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:34.173716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c432c20-46ea-4045-8ed1-caae75f4c87e admin admin] PATH_INFO: `/v3/projects/48ad2f7d710548259e5ddcf70a7d213b` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:34.175511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c432c20-46ea-4045-8ed1-caae75f4c87e admin admin] RBAC: Authorizing `identity:delete_project(project_id=48ad2f7d710548259e5ddcf70a7d213b)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:34.177726 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c432c20-46ea-4045-8ed1-caae75f4c87e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:34.238527 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1c432c20-46ea-4045-8ed1-caae75f4c87e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48ad2f7d710548259e5ddcf70a7d213b', 'request_id': 'req-1c432c20-46ea-4045-8ed1-caae75f4c87e'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:34.246407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1c432c20-46ea-4045-8ed1-caae75f4c87e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48ad2f7d710548259e5ddcf70a7d213b', 'request_id': 'req-1c432c20-46ea-4045-8ed1-caae75f4c87e'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:34.254113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1c432c20-46ea-4045-8ed1-caae75f4c87e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48ad2f7d710548259e5ddcf70a7d213b', 'request_id': 'req-1c432c20-46ea-4045-8ed1-caae75f4c87e'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:34.265268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1c432c20-46ea-4045-8ed1-caae75f4c87e admin admin] The token cache is being invalidate because project 48ad2f7d710548259e5ddcf70a7d213b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:34.265583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1c432c20-46ea-4045-8ed1-caae75f4c87e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:34.266272 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1c432c20-46ea-4045-8ed1-caae75f4c87e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:34.267899 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 964/1903] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:34 2022] DELETE /identity/v3/projects/48ad2f7d710548259e5ddcf70a7d213b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:34.272556 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4327d-5a68-4ec0-920d-4592a135e94c service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:34.282836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4327d-5a68-4ec0-920d-4592a135e94c service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:34.350340 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6220437-1578-4c83-9ba3-588f1b8d956a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:34.351238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6220437-1578-4c83-9ba3-588f1b8d956a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:34.351530 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6220437-1578-4c83-9ba3-588f1b8d956a admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:34.351807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6220437-1578-4c83-9ba3-588f1b8d956a admin admin] PATH_INFO: `/v3/users/294c86c679d94f56979ec6961c69c235` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:34.353091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6220437-1578-4c83-9ba3-588f1b8d956a admin admin] RBAC: Authorizing `identity:delete_user(user_id=294c86c679d94f56979ec6961c69c235)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:34.355130 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6220437-1578-4c83-9ba3-588f1b8d956a admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:34.399259 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:34.409753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e6220437-1578-4c83-9ba3-588f1b8d956a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '294c86c679d94f56979ec6961c69c235'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:34.425502 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e6220437-1578-4c83-9ba3-588f1b8d956a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '294c86c679d94f56979ec6961c69c235'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:34.432883 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e6220437-1578-4c83-9ba3-588f1b8d956a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '294c86c679d94f56979ec6961c69c235'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:34.433614 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e6220437-1578-4c83-9ba3-588f1b8d956a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '294c86c679d94f56979ec6961c69c235'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:34.441307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e6220437-1578-4c83-9ba3-588f1b8d956a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '294c86c679d94f56979ec6961c69c235'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:34.447396 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 940/1904] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:34 2022] DELETE /identity/v3/users/294c86c679d94f56979ec6961c69c235 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:34.802562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c432c20-46ea-4045-8ed1-caae75f4c87e admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:34.804117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1c432c20-46ea-4045-8ed1-caae75f4c87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:34.826618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c432c20-46ea-4045-8ed1-caae75f4c87e admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:34.888333 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ecfe4b-124b-404a-8690-939ae72f3004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:34.889009 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecfe4b-124b-404a-8690-939ae72f3004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:34.889288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecfe4b-124b-404a-8690-939ae72f3004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:34.889560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ecfe4b-124b-404a-8690-939ae72f3004 admin admin] PATH_INFO: `/v3/projects/0c4203b821c142859d098893f015fcd7` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:34.890746 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ecfe4b-124b-404a-8690-939ae72f3004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c4203b821c142859d098893f015fcd7)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:34.892619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ecfe4b-124b-404a-8690-939ae72f3004 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:34.923777 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:34.940888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-44ecfe4b-124b-404a-8690-939ae72f3004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c4203b821c142859d098893f015fcd7', 'request_id': 'req-44ecfe4b-124b-404a-8690-939ae72f3004'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:34.947962 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-44ecfe4b-124b-404a-8690-939ae72f3004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c4203b821c142859d098893f015fcd7', 'request_id': 'req-44ecfe4b-124b-404a-8690-939ae72f3004'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:34.954099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-44ecfe4b-124b-404a-8690-939ae72f3004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c4203b821c142859d098893f015fcd7', 'request_id': 'req-44ecfe4b-124b-404a-8690-939ae72f3004'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:34.964736 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-44ecfe4b-124b-404a-8690-939ae72f3004 admin admin] The token cache is being invalidate because project 0c4203b821c142859d098893f015fcd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:34.965014 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-44ecfe4b-124b-404a-8690-939ae72f3004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:34.965633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-44ecfe4b-124b-404a-8690-939ae72f3004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:34.967156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 965/1905] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:34 2022] DELETE /identity/v3/projects/0c4203b821c142859d098893f015fcd7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:53:34.971136 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6220437-1578-4c83-9ba3-588f1b8d956a admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:34.972603 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e6220437-1578-4c83-9ba3-588f1b8d956a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:34.985302 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6220437-1578-4c83-9ba3-588f1b8d956a admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:35.040339 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54e810-8627-4aac-9816-b0f40b9ab070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:35.040976 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54e810-8627-4aac-9816-b0f40b9ab070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:35.041257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54e810-8627-4aac-9816-b0f40b9ab070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:35.041527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54e810-8627-4aac-9816-b0f40b9ab070 admin admin] PATH_INFO: `/v3/users/aa2b01b19de74c4bba4fc76b0d5e0549` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:35.042696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54e810-8627-4aac-9816-b0f40b9ab070 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa2b01b19de74c4bba4fc76b0d5e0549)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:35.044623 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54e810-8627-4aac-9816-b0f40b9ab070 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:35.089213 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8f54e810-8627-4aac-9816-b0f40b9ab070 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa2b01b19de74c4bba4fc76b0d5e0549'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:35.100889 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8f54e810-8627-4aac-9816-b0f40b9ab070 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa2b01b19de74c4bba4fc76b0d5e0549'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:35.107941 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8f54e810-8627-4aac-9816-b0f40b9ab070 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa2b01b19de74c4bba4fc76b0d5e0549'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:35.108578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8f54e810-8627-4aac-9816-b0f40b9ab070 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa2b01b19de74c4bba4fc76b0d5e0549'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:35.116941 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8f54e810-8627-4aac-9816-b0f40b9ab070 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa2b01b19de74c4bba4fc76b0d5e0549'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:35.122938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 941/1906] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:34 2022] DELETE /identity/v3/users/aa2b01b19de74c4bba4fc76b0d5e0549 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:35.418704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ecfe4b-124b-404a-8690-939ae72f3004 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:35.420114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-44ecfe4b-124b-404a-8690-939ae72f3004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:35.433755 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ecfe4b-124b-404a-8690-939ae72f3004 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:35.493054 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588bb7d7-dd94-4b4a-95fa-4b2b829f0e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:35.493637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bb7d7-dd94-4b4a-95fa-4b2b829f0e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:35.493917 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bb7d7-dd94-4b4a-95fa-4b2b829f0e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:35.494190 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588bb7d7-dd94-4b4a-95fa-4b2b829f0e1d admin admin] PATH_INFO: `/v3/projects/b01d3d1488944e4a9814b91cc1785dfd` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:35.495386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bb7d7-dd94-4b4a-95fa-4b2b829f0e1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b01d3d1488944e4a9814b91cc1785dfd)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:35.497284 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588bb7d7-dd94-4b4a-95fa-4b2b829f0e1d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:35.541486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-588bb7d7-dd94-4b4a-95fa-4b2b829f0e1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b01d3d1488944e4a9814b91cc1785dfd', 'request_id': 'req-588bb7d7-dd94-4b4a-95fa-4b2b829f0e1d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:35.547978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-588bb7d7-dd94-4b4a-95fa-4b2b829f0e1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b01d3d1488944e4a9814b91cc1785dfd', 'request_id': 'req-588bb7d7-dd94-4b4a-95fa-4b2b829f0e1d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:35.554950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-588bb7d7-dd94-4b4a-95fa-4b2b829f0e1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b01d3d1488944e4a9814b91cc1785dfd', 'request_id': 'req-588bb7d7-dd94-4b4a-95fa-4b2b829f0e1d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:35.565156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-588bb7d7-dd94-4b4a-95fa-4b2b829f0e1d admin admin] The token cache is being invalidate because project b01d3d1488944e4a9814b91cc1785dfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:35.565443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-588bb7d7-dd94-4b4a-95fa-4b2b829f0e1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:35.566075 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-588bb7d7-dd94-4b4a-95fa-4b2b829f0e1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:35.567599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 966/1907] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:35 2022] DELETE /identity/v3/projects/b01d3d1488944e4a9814b91cc1785dfd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:35.573973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a532ca-50dc-401c-948b-348b66bf766e None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:35.574256 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a532ca-50dc-401c-948b-348b66bf766e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:35.574562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a532ca-50dc-401c-948b-348b66bf766e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:35.593370 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-63a532ca-50dc-401c-948b-348b66bf766e None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:35.594611 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-63a532ca-50dc-401c-948b-348b66bf766e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:35.620514 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:35.657723 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 942/1908] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:35.666275 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c1f0c-667f-4775-85a6-ab4be4d80a5d None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:35.666590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c1f0c-667f-4775-85a6-ab4be4d80a5d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:35.666870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c1f0c-667f-4775-85a6-ab4be4d80a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:35.688818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-899c1f0c-667f-4775-85a6-ab4be4d80a5d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:35.690343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-899c1f0c-667f-4775-85a6-ab4be4d80a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:35.702889 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 967/1909] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:35.706982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a532ca-50dc-401c-948b-348b66bf766e None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:35.717450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a532ca-50dc-401c-948b-348b66bf766e None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:35.722607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0008d9e-3bb4-4fcf-ab0a-1f27f49c3f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:35.723226 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0008d9e-3bb4-4fcf-ab0a-1f27f49c3f89 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:35.723533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0008d9e-3bb4-4fcf-ab0a-1f27f49c3f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:35.723809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0008d9e-3bb4-4fcf-ab0a-1f27f49c3f89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:35.724661 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0008d9e-3bb4-4fcf-ab0a-1f27f49c3f89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:53:35.724972 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0008d9e-3bb4-4fcf-ab0a-1f27f49c3f89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:35.726628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0008d9e-3bb4-4fcf-ab0a-1f27f49c3f89 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:35.739380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 943/1910] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:53:35.744048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c1f0c-667f-4775-85a6-ab4be4d80a5d None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:35.753763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c1f0c-667f-4775-85a6-ab4be4d80a5d None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:35.758527 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33274d-e22c-47af-b6d6-7709c012a606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:35.759130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33274d-e22c-47af-b6d6-7709c012a606 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:35.759405 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33274d-e22c-47af-b6d6-7709c012a606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:35.759672 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33274d-e22c-47af-b6d6-7709c012a606 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:35.760341 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33274d-e22c-47af-b6d6-7709c012a606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:35.762241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33274d-e22c-47af-b6d6-7709c012a606 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:35.790725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 968/1911] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:35 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:35.794888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0008d9e-3bb4-4fcf-ab0a-1f27f49c3f89 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:35.806475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0008d9e-3bb4-4fcf-ab0a-1f27f49c3f89 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:35.885164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9072e1-3ce0-4e4d-bfc1-3631ac48cf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:35.885962 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9072e1-3ce0-4e4d-bfc1-3631ac48cf93 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:35.886315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9072e1-3ce0-4e4d-bfc1-3631ac48cf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:35.886628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9072e1-3ce0-4e4d-bfc1-3631ac48cf93 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:35.888060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9072e1-3ce0-4e4d-bfc1-3631ac48cf93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:35.897824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9072e1-3ce0-4e4d-bfc1-3631ac48cf93 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:35.929316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 944/1912] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:35 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:35.933289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33274d-e22c-47af-b6d6-7709c012a606 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:35.964210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33274d-e22c-47af-b6d6-7709c012a606 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:35.976499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d19a40-8a3b-4727-9dd3-f8bfa9d06c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:35.977155 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d19a40-8a3b-4727-9dd3-f8bfa9d06c2b admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:35.977424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d19a40-8a3b-4727-9dd3-f8bfa9d06c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:35.977703 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d19a40-8a3b-4727-9dd3-f8bfa9d06c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:35.978770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d19a40-8a3b-4727-9dd3-f8bfa9d06c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:35.980532 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d19a40-8a3b-4727-9dd3-f8bfa9d06c2b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:35.998735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 969/1913] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:35.999939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9072e1-3ce0-4e4d-bfc1-3631ac48cf93 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:36.024269 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9072e1-3ce0-4e4d-bfc1-3631ac48cf93 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:36.029792 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35969732-eafc-4cfa-bc48-cfcfe0be8f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:36.030354 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35969732-eafc-4cfa-bc48-cfcfe0be8f9a admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:36.030775 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35969732-eafc-4cfa-bc48-cfcfe0be8f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:36.032845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35969732-eafc-4cfa-bc48-cfcfe0be8f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:36.034075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35969732-eafc-4cfa-bc48-cfcfe0be8f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:36.038621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35969732-eafc-4cfa-bc48-cfcfe0be8f9a admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:36.053299 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 945/1914] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:36.057504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d19a40-8a3b-4727-9dd3-f8bfa9d06c2b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:36.064602 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:36.071417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d19a40-8a3b-4727-9dd3-f8bfa9d06c2b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:36.077251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337aa36-523c-4949-90f4-b64e05fa1616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:36.078062 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337aa36-523c-4949-90f4-b64e05fa1616 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:36.078451 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337aa36-523c-4949-90f4-b64e05fa1616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:36.078830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337aa36-523c-4949-90f4-b64e05fa1616 admin admin] PATH_INFO: `/v3/projects/493dcdf9660d44d0a116a94bb50085bd/users/217711bb1be940f292d5c8e7a51d85d9/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:36.111256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337aa36-523c-4949-90f4-b64e05fa1616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=493dcdf9660d44d0a116a94bb50085bd, user_id=217711bb1be940f292d5c8e7a51d85d9, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:36.117286 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337aa36-523c-4949-90f4-b64e05fa1616 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:36.148028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 970/1915] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:36 2022] PUT /identity/v3/projects/493dcdf9660d44d0a116a94bb50085bd/users/217711bb1be940f292d5c8e7a51d85d9/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:36.160988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b2d76-4df5-4329-93ac-5fe91d5df711 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:36.161276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b2d76-4df5-4329-93ac-5fe91d5df711 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:36.161552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b2d76-4df5-4329-93ac-5fe91d5df711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:36.187895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14497c-54a5-4489-a39e-e16f64964ee8 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:36.188193 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14497c-54a5-4489-a39e-e16f64964ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:36.188472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14497c-54a5-4489-a39e-e16f64964ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:36.210819 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-ac14497c-54a5-4489-a39e-e16f64964ee8 None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:36.213170 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ac14497c-54a5-4489-a39e-e16f64964ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:36.215703 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-5d5b2d76-4df5-4329-93ac-5fe91d5df711 None None] MFA Rules not processed for user `217711bb1be940f292d5c8e7a51d85d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:36.216990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-5d5b2d76-4df5-4329-93ac-5fe91d5df711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:36.290215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 971/1916] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 3 12:53:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:53:36.294239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 946/1917] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:36.306790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14497c-54a5-4489-a39e-e16f64964ee8 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:36.308153 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ac14497c-54a5-4489-a39e-e16f64964ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:36.311708 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b2d76-4df5-4329-93ac-5fe91d5df711 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:36.313140 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-5d5b2d76-4df5-4329-93ac-5fe91d5df711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:36.417243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:36.418706 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14497c-54a5-4489-a39e-e16f64964ee8 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:36.430629 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5f8dd-9eb9-4cd2-b841-620d845070f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:36.432050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5f8dd-9eb9-4cd2-b841-620d845070f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:36.434130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5f8dd-9eb9-4cd2-b841-620d845070f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:36.434714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5f8dd-9eb9-4cd2-b841-620d845070f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:36.456811 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5f8dd-9eb9-4cd2-b841-620d845070f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:36.458421 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5f8dd-9eb9-4cd2-b841-620d845070f5 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:36.483022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 972/1918] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:36.529386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b2d76-4df5-4329-93ac-5fe91d5df711 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:36.557877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151b690-b2d8-4afb-9f88-c29e514f4ccb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:36.558561 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151b690-b2d8-4afb-9f88-c29e514f4ccb service swift] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:36.558848 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151b690-b2d8-4afb-9f88-c29e514f4ccb service swift] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:36.559123 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151b690-b2d8-4afb-9f88-c29e514f4ccb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:36.579966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151b690-b2d8-4afb-9f88-c29e514f4ccb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:36.582732 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151b690-b2d8-4afb-9f88-c29e514f4ccb service swift] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:36.592661 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 947/1919] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:53:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 03 12:53:38.998062 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5f8dd-9eb9-4cd2-b841-620d845070f5 service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:38.999574 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-80f5f8dd-9eb9-4cd2-b841-620d845070f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:39.058569 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5f8dd-9eb9-4cd2-b841-620d845070f5 service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:39.062191 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac10add-6676-49a8-8d55-88fd7d1edd7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:39.062842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10add-6676-49a8-8d55-88fd7d1edd7d service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:39.063175 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10add-6676-49a8-8d55-88fd7d1edd7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:39.063397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac10add-6676-49a8-8d55-88fd7d1edd7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:39.065775 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8ac10add-6676-49a8-8d55-88fd7d1edd7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:39.075943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac10add-6676-49a8-8d55-88fd7d1edd7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:39.077350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac10add-6676-49a8-8d55-88fd7d1edd7d service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:39.088840 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 973/1920] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:40.441962 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b96671-2801-42fd-918c-664bd8de61c0 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:40.442456 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b96671-2801-42fd-918c-664bd8de61c0 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:40.442578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b96671-2801-42fd-918c-664bd8de61c0 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:40.444451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 948/1921] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 3 12:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:53:40.499840 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa2cb5-2dae-421b-af18-46b601ffe5c4 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:40.500353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa2cb5-2dae-421b-af18-46b601ffe5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:40.500467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa2cb5-2dae-421b-af18-46b601ffe5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:40.556708 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-07aa2cb5-2dae-421b-af18-46b601ffe5c4 None None] MFA Rules not processed for user `7b634f567dca42dc95df175a4f36c1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:40.558037 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-07aa2cb5-2dae-421b-af18-46b601ffe5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:40.591616 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:40.664226 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 974/1922] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 3 12:53:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:40.694158 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b96671-2801-42fd-918c-664bd8de61c0 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:40.695961 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-09b96671-2801-42fd-918c-664bd8de61c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:40.785561 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b96671-2801-42fd-918c-664bd8de61c0 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:40.799890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd80e7-f583-4758-a6bc-c0c5c1f484cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:40.802995 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd80e7-f583-4758-a6bc-c0c5c1f484cf service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:40.803417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd80e7-f583-4758-a6bc-c0c5c1f484cf service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:40.803850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd80e7-f583-4758-a6bc-c0c5c1f484cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:40.828921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd80e7-f583-4758-a6bc-c0c5c1f484cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:40.831238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd80e7-f583-4758-a6bc-c0c5c1f484cf service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:40.846709 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 949/1923] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:41.380892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450bcde-df5b-4a91-8548-cd7839389a08 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:41.381526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450bcde-df5b-4a91-8548-cd7839389a08 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:41.381990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450bcde-df5b-4a91-8548-cd7839389a08 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:41.384521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 975/1924] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 3 12:53:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:53:41.395248 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877f10d-9727-42ce-be24-257d793c58bf None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:41.395546 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877f10d-9727-42ce-be24-257d793c58bf None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:41.395826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877f10d-9727-42ce-be24-257d793c58bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:41.417174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-c877f10d-9727-42ce-be24-257d793c58bf None None] MFA Rules not processed for user `7b634f567dca42dc95df175a4f36c1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:41.418639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c877f10d-9727-42ce-be24-257d793c58bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:41.429980 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 950/1925] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 3 12:53:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:41.447245 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450bcde-df5b-4a91-8548-cd7839389a08 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:41.449763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-9450bcde-df5b-4a91-8548-cd7839389a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:41.475887 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450bcde-df5b-4a91-8548-cd7839389a08 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:41.484344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55fc3e0-5753-4151-b292-ae822f852bf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:41.485057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fc3e0-5753-4151-b292-ae822f852bf0 service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:41.485425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fc3e0-5753-4151-b292-ae822f852bf0 service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:41.485707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55fc3e0-5753-4151-b292-ae822f852bf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:41.499572 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55fc3e0-5753-4151-b292-ae822f852bf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:41.501293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55fc3e0-5753-4151-b292-ae822f852bf0 service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:41.529437 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 976/1926] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:47.590143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877f10d-9727-42ce-be24-257d793c58bf None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:47.591583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c877f10d-9727-42ce-be24-257d793c58bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:47.656511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877f10d-9727-42ce-be24-257d793c58bf None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:47.660946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5b88d-f17f-4265-b8fd-dbf7686c0aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:47.661577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5b88d-f17f-4265-b8fd-dbf7686c0aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:47.661859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5b88d-f17f-4265-b8fd-dbf7686c0aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:47.662146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5b88d-f17f-4265-b8fd-dbf7686c0aeb admin admin] PATH_INFO: `/v3/users/8afe2e9d859246ca85ce3f824b2cace1` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:47.663400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5b88d-f17f-4265-b8fd-dbf7686c0aeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8afe2e9d859246ca85ce3f824b2cace1)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:47.665287 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5b88d-f17f-4265-b8fd-dbf7686c0aeb admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:47.715432 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-35e5b88d-f17f-4265-b8fd-dbf7686c0aeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8afe2e9d859246ca85ce3f824b2cace1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:47.729344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-35e5b88d-f17f-4265-b8fd-dbf7686c0aeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8afe2e9d859246ca85ce3f824b2cace1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:47.738112 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-35e5b88d-f17f-4265-b8fd-dbf7686c0aeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8afe2e9d859246ca85ce3f824b2cace1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:47.738879 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-35e5b88d-f17f-4265-b8fd-dbf7686c0aeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8afe2e9d859246ca85ce3f824b2cace1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:47.746543 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-35e5b88d-f17f-4265-b8fd-dbf7686c0aeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8afe2e9d859246ca85ce3f824b2cace1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:47.752625 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 951/1927] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:47 2022] DELETE /identity/v3/users/8afe2e9d859246ca85ce3f824b2cace1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:47.758565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bc7a2-9f6a-4927-be82-b58fb28fc9ae None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:47.759106 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bc7a2-9f6a-4927-be82-b58fb28fc9ae None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:47.759606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bc7a2-9f6a-4927-be82-b58fb28fc9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:47.785462 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-902bc7a2-9f6a-4927-be82-b58fb28fc9ae None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:47.786769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-902bc7a2-9f6a-4927-be82-b58fb28fc9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:47.890217 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 977/1928] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:47.943490 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5b88d-f17f-4265-b8fd-dbf7686c0aeb admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:47.945010 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-35e5b88d-f17f-4265-b8fd-dbf7686c0aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:47.965531 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:48.014634 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5b88d-f17f-4265-b8fd-dbf7686c0aeb admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:48.030728 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f1bb7-7982-41d5-8c3e-444d4bc2457e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:48.031291 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f1bb7-7982-41d5-8c3e-444d4bc2457e service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:48.031822 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f1bb7-7982-41d5-8c3e-444d4bc2457e service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:48.032039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f1bb7-7982-41d5-8c3e-444d4bc2457e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:48.043886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f1bb7-7982-41d5-8c3e-444d4bc2457e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:48.045387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f1bb7-7982-41d5-8c3e-444d4bc2457e service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:48.056304 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 952/1929] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:48.539727 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902bc7a2-9f6a-4927-be82-b58fb28fc9ae None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:48.541331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-902bc7a2-9f6a-4927-be82-b58fb28fc9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:48.560148 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902bc7a2-9f6a-4927-be82-b58fb28fc9ae None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:48.573558 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bddda-abb1-40e7-a826-20b0c39afda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:48.574572 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bddda-abb1-40e7-a826-20b0c39afda2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:48.575058 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bddda-abb1-40e7-a826-20b0c39afda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:48.575546 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bddda-abb1-40e7-a826-20b0c39afda2 admin admin] PATH_INFO: `/v3/users/217711bb1be940f292d5c8e7a51d85d9` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:48.577092 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072bddda-abb1-40e7-a826-20b0c39afda2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=217711bb1be940f292d5c8e7a51d85d9)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:48.580726 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072bddda-abb1-40e7-a826-20b0c39afda2 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:48.715443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-072bddda-abb1-40e7-a826-20b0c39afda2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '217711bb1be940f292d5c8e7a51d85d9'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:48.729471 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-072bddda-abb1-40e7-a826-20b0c39afda2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '217711bb1be940f292d5c8e7a51d85d9'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:48.757564 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-072bddda-abb1-40e7-a826-20b0c39afda2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '217711bb1be940f292d5c8e7a51d85d9'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:48.759732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-072bddda-abb1-40e7-a826-20b0c39afda2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '217711bb1be940f292d5c8e7a51d85d9'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:48.762698 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-072bddda-abb1-40e7-a826-20b0c39afda2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '217711bb1be940f292d5c8e7a51d85d9'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:48.789764 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 978/1930] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:48 2022] DELETE /identity/v3/users/217711bb1be940f292d5c8e7a51d85d9 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:48.793956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac87d21-e1f4-4e2f-b457-5c5578e05693 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:48.794450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac87d21-e1f4-4e2f-b457-5c5578e05693 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:48.794693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac87d21-e1f4-4e2f-b457-5c5578e05693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:48.816551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-8ac87d21-e1f4-4e2f-b457-5c5578e05693 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:48.818804 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8ac87d21-e1f4-4e2f-b457-5c5578e05693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:48.914960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 953/1931] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:53:48.924886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bddda-abb1-40e7-a826-20b0c39afda2 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:48.926916 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-072bddda-abb1-40e7-a826-20b0c39afda2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:48.961614 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:49.004863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bddda-abb1-40e7-a826-20b0c39afda2 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:49.013431 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad0cee-17c5-4291-b819-511349e475fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:49.014159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad0cee-17c5-4291-b819-511349e475fd service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:49.014553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad0cee-17c5-4291-b819-511349e475fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:49.014832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad0cee-17c5-4291-b819-511349e475fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:49.023047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac87d21-e1f4-4e2f-b457-5c5578e05693 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:49.025059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8ac87d21-e1f4-4e2f-b457-5c5578e05693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:49.027676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad0cee-17c5-4291-b819-511349e475fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:49.029301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad0cee-17c5-4291-b819-511349e475fd service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:49.037408 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac87d21-e1f4-4e2f-b457-5c5578e05693 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:49.041493 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 979/1932] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:49.042348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f14a44-2763-4ecd-ae96-7a767fa59ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:49.043104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f14a44-2763-4ecd-ae96-7a767fa59ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:49.043388 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f14a44-2763-4ecd-ae96-7a767fa59ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:49.043661 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f14a44-2763-4ecd-ae96-7a767fa59ba0 admin admin] PATH_INFO: `/v3/projects/351a17f93f4c42b09f319a49a711a9cc` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:49.045544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f14a44-2763-4ecd-ae96-7a767fa59ba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=351a17f93f4c42b09f319a49a711a9cc)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:49.047533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f14a44-2763-4ecd-ae96-7a767fa59ba0 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:49.103816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-96f14a44-2763-4ecd-ae96-7a767fa59ba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '351a17f93f4c42b09f319a49a711a9cc', 'request_id': 'req-96f14a44-2763-4ecd-ae96-7a767fa59ba0'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.111009 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-96f14a44-2763-4ecd-ae96-7a767fa59ba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '351a17f93f4c42b09f319a49a711a9cc', 'request_id': 'req-96f14a44-2763-4ecd-ae96-7a767fa59ba0'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.117719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-96f14a44-2763-4ecd-ae96-7a767fa59ba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '351a17f93f4c42b09f319a49a711a9cc', 'request_id': 'req-96f14a44-2763-4ecd-ae96-7a767fa59ba0'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.130023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-96f14a44-2763-4ecd-ae96-7a767fa59ba0 admin admin] The token cache is being invalidate because project 351a17f93f4c42b09f319a49a711a9cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:49.130313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-96f14a44-2763-4ecd-ae96-7a767fa59ba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.131055 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-96f14a44-2763-4ecd-ae96-7a767fa59ba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.132618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 954/1933] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:49 2022] DELETE /identity/v3/projects/351a17f93f4c42b09f319a49a711a9cc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:53:49.136646 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad0cee-17c5-4291-b819-511349e475fd service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:49.149414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad0cee-17c5-4291-b819-511349e475fd service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:49.229890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f88b8d-7b20-4aa4-a56f-5ccafa7ca9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:49.230708 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f88b8d-7b20-4aa4-a56f-5ccafa7ca9d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:49.230999 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f88b8d-7b20-4aa4-a56f-5ccafa7ca9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:49.231344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f88b8d-7b20-4aa4-a56f-5ccafa7ca9d9 admin admin] PATH_INFO: `/v3/users/3bd7338ec96547139b07b3affe60f507` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:49.232811 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f88b8d-7b20-4aa4-a56f-5ccafa7ca9d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bd7338ec96547139b07b3affe60f507)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:49.234925 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f88b8d-7b20-4aa4-a56f-5ccafa7ca9d9 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:49.286017 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-55f88b8d-7b20-4aa4-a56f-5ccafa7ca9d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bd7338ec96547139b07b3affe60f507'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.294150 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-55f88b8d-7b20-4aa4-a56f-5ccafa7ca9d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bd7338ec96547139b07b3affe60f507'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.309454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-55f88b8d-7b20-4aa4-a56f-5ccafa7ca9d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bd7338ec96547139b07b3affe60f507'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.310731 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-55f88b8d-7b20-4aa4-a56f-5ccafa7ca9d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bd7338ec96547139b07b3affe60f507'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.311493 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-55f88b8d-7b20-4aa4-a56f-5ccafa7ca9d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bd7338ec96547139b07b3affe60f507'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.335855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 980/1934] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:49 2022] DELETE /identity/v3/users/3bd7338ec96547139b07b3affe60f507 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:49.354690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f14a44-2763-4ecd-ae96-7a767fa59ba0 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:49.356511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-96f14a44-2763-4ecd-ae96-7a767fa59ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:49.398380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f14a44-2763-4ecd-ae96-7a767fa59ba0 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:49.451189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:49.469833 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fcf13b-ff24-44d5-8605-c8d4049dbe23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:49.470492 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fcf13b-ff24-44d5-8605-c8d4049dbe23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:49.470805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fcf13b-ff24-44d5-8605-c8d4049dbe23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:49.471080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fcf13b-ff24-44d5-8605-c8d4049dbe23 admin admin] PATH_INFO: `/v3/projects/493dcdf9660d44d0a116a94bb50085bd` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:49.472813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fcf13b-ff24-44d5-8605-c8d4049dbe23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=493dcdf9660d44d0a116a94bb50085bd)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:49.474840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fcf13b-ff24-44d5-8605-c8d4049dbe23 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:49.526382 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-52fcf13b-ff24-44d5-8605-c8d4049dbe23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '493dcdf9660d44d0a116a94bb50085bd', 'request_id': 'req-52fcf13b-ff24-44d5-8605-c8d4049dbe23'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.534836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-52fcf13b-ff24-44d5-8605-c8d4049dbe23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '493dcdf9660d44d0a116a94bb50085bd', 'request_id': 'req-52fcf13b-ff24-44d5-8605-c8d4049dbe23'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.541848 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-52fcf13b-ff24-44d5-8605-c8d4049dbe23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '493dcdf9660d44d0a116a94bb50085bd', 'request_id': 'req-52fcf13b-ff24-44d5-8605-c8d4049dbe23'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.552912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-52fcf13b-ff24-44d5-8605-c8d4049dbe23 admin admin] The token cache is being invalidate because project 493dcdf9660d44d0a116a94bb50085bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:49.553192 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-52fcf13b-ff24-44d5-8605-c8d4049dbe23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.553836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-52fcf13b-ff24-44d5-8605-c8d4049dbe23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.555656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 955/1935] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:49 2022] DELETE /identity/v3/projects/493dcdf9660d44d0a116a94bb50085bd => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:49.561124 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cd0c7-b030-40d1-82c5-0455a208f1f1 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:49.561487 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cd0c7-b030-40d1-82c5-0455a208f1f1 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:49.561800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cd0c7-b030-40d1-82c5-0455a208f1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:49.585425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-d58cd0c7-b030-40d1-82c5-0455a208f1f1 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:49.586725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d58cd0c7-b030-40d1-82c5-0455a208f1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:49.632050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:49.648913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fcf13b-ff24-44d5-8605-c8d4049dbe23 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:49.650434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-52fcf13b-ff24-44d5-8605-c8d4049dbe23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:49.672787 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 981/1936] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:49.681125 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b206705-dd7f-44e6-838b-64c63c478303 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:49.681475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b206705-dd7f-44e6-838b-64c63c478303 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:49.681692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b206705-dd7f-44e6-838b-64c63c478303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:49.715907 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1b206705-dd7f-44e6-838b-64c63c478303 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:49.717125 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1b206705-dd7f-44e6-838b-64c63c478303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:49.727872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 982/1937] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:49.732300 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b206705-dd7f-44e6-838b-64c63c478303 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:49.742336 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b206705-dd7f-44e6-838b-64c63c478303 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:49.747246 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300c0d9-d373-4aae-ac35-69680d244674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:49.747937 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300c0d9-d373-4aae-ac35-69680d244674 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:49.748308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300c0d9-d373-4aae-ac35-69680d244674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:49.748890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300c0d9-d373-4aae-ac35-69680d244674 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:49.749772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300c0d9-d373-4aae-ac35-69680d244674 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:53:49.750199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300c0d9-d373-4aae-ac35-69680d244674 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:49.752023 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300c0d9-d373-4aae-ac35-69680d244674 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:49.752814 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fcf13b-ff24-44d5-8605-c8d4049dbe23 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:49.757875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe15341-6eb0-4691-9094-083f71e0527b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:49.758667 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe15341-6eb0-4691-9094-083f71e0527b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:49.759023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe15341-6eb0-4691-9094-083f71e0527b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:49.759375 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe15341-6eb0-4691-9094-083f71e0527b admin admin] PATH_INFO: `/v3/projects/5ba0ff4745be4cd78e6ba900c724b624` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:49.760637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe15341-6eb0-4691-9094-083f71e0527b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba0ff4745be4cd78e6ba900c724b624)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:49.762733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe15341-6eb0-4691-9094-083f71e0527b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:49.774353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 983/1938] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:53:49.775359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300c0d9-d373-4aae-ac35-69680d244674 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:49.788401 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300c0d9-d373-4aae-ac35-69680d244674 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:49.793132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6ab84-6a1b-4ca5-868e-d76a0ea7bf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:49.793933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6ab84-6a1b-4ca5-868e-d76a0ea7bf21 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:49.794432 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6ab84-6a1b-4ca5-868e-d76a0ea7bf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:49.794745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6ab84-6a1b-4ca5-868e-d76a0ea7bf21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:49.795441 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6ab84-6a1b-4ca5-868e-d76a0ea7bf21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:49.797364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6ab84-6a1b-4ca5-868e-d76a0ea7bf21 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:49.827771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0fe15341-6eb0-4691-9094-083f71e0527b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba0ff4745be4cd78e6ba900c724b624', 'request_id': 'req-0fe15341-6eb0-4691-9094-083f71e0527b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.829637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 984/1939] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:49 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:49.833896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6ab84-6a1b-4ca5-868e-d76a0ea7bf21 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:49.839209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0fe15341-6eb0-4691-9094-083f71e0527b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba0ff4745be4cd78e6ba900c724b624', 'request_id': 'req-0fe15341-6eb0-4691-9094-083f71e0527b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.847038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0fe15341-6eb0-4691-9094-083f71e0527b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba0ff4745be4cd78e6ba900c724b624', 'request_id': 'req-0fe15341-6eb0-4691-9094-083f71e0527b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.858701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0fe15341-6eb0-4691-9094-083f71e0527b admin admin] The token cache is being invalidate because project 5ba0ff4745be4cd78e6ba900c724b624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:49.859111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0fe15341-6eb0-4691-9094-083f71e0527b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.859795 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0fe15341-6eb0-4691-9094-083f71e0527b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:49.861738 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 956/1940] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:49 2022] DELETE /identity/v3/projects/5ba0ff4745be4cd78e6ba900c724b624 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:49.868322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c5100-11b1-45de-bd3d-5ed62e4141a9 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:49.868653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c5100-11b1-45de-bd3d-5ed62e4141a9 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:49.868850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c5100-11b1-45de-bd3d-5ed62e4141a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:49.876708 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:49.896358 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-c06c5100-11b1-45de-bd3d-5ed62e4141a9 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:49.899056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c06c5100-11b1-45de-bd3d-5ed62e4141a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:49.916407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6ab84-6a1b-4ca5-868e-d76a0ea7bf21 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:49.922048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396e5eb-a2db-4539-803f-1f8ef9049423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:49.922826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396e5eb-a2db-4539-803f-1f8ef9049423 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:49.923111 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396e5eb-a2db-4539-803f-1f8ef9049423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:49.923386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396e5eb-a2db-4539-803f-1f8ef9049423 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:49.924058 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396e5eb-a2db-4539-803f-1f8ef9049423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:49.927140 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396e5eb-a2db-4539-803f-1f8ef9049423 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:49.951780 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 985/1941] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:49 2022] POST /identity/v3/users => generated 286 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:53:49.955923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396e5eb-a2db-4539-803f-1f8ef9049423 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:49.958349 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0396e5eb-a2db-4539-803f-1f8ef9049423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:49.970455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 957/1942] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:53:49.972160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396e5eb-a2db-4539-803f-1f8ef9049423 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:49.978131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a255a-3e90-4578-9a29-2ef02890b202 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:49.978425 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a255a-3e90-4578-9a29-2ef02890b202 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:49.978736 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a255a-3e90-4578-9a29-2ef02890b202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:49.988265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b20d2e-19b9-44ca-8822-aab1f75f65f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:49.988480 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b20d2e-19b9-44ca-8822-aab1f75f65f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:49.988718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b20d2e-19b9-44ca-8822-aab1f75f65f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:49.988992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b20d2e-19b9-44ca-8822-aab1f75f65f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:49.989700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b20d2e-19b9-44ca-8822-aab1f75f65f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:49.991345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b20d2e-19b9-44ca-8822-aab1f75f65f9 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:50.005823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 986/1943] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:53:50.009518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b20d2e-19b9-44ca-8822-aab1f75f65f9 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:50.021829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-d19a255a-3e90-4578-9a29-2ef02890b202 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:50.022200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b20d2e-19b9-44ca-8822-aab1f75f65f9 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:50.023563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d19a255a-3e90-4578-9a29-2ef02890b202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:50.029650 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8462748-779c-480a-b62f-6db5e710b669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:50.030988 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8462748-779c-480a-b62f-6db5e710b669 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:50.031492 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8462748-779c-480a-b62f-6db5e710b669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:50.032005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8462748-779c-480a-b62f-6db5e710b669 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:50.033084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8462748-779c-480a-b62f-6db5e710b669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:50.034527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 958/1944] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:53:50.036229 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8462748-779c-480a-b62f-6db5e710b669 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:50.037486 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a255a-3e90-4578-9a29-2ef02890b202 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:50.047739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 987/1945] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:53:50.052884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8462748-779c-480a-b62f-6db5e710b669 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:50.053455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a255a-3e90-4578-9a29-2ef02890b202 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:50.059695 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ce86f-8cab-4d8a-ade7-b236b9af90d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:50.060328 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ce86f-8cab-4d8a-ade7-b236b9af90d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:50.060894 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ce86f-8cab-4d8a-ade7-b236b9af90d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:50.061178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ce86f-8cab-4d8a-ade7-b236b9af90d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:50.062023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ce86f-8cab-4d8a-ade7-b236b9af90d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:53:50.062339 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ce86f-8cab-4d8a-ade7-b236b9af90d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:50.062782 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8462748-779c-480a-b62f-6db5e710b669 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:50.063973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ce86f-8cab-4d8a-ade7-b236b9af90d2 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:50.068082 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23edc0-d8f5-4329-b922-fa8704fd0322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:50.068748 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23edc0-d8f5-4329-b922-fa8704fd0322 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:50.069034 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23edc0-d8f5-4329-b922-fa8704fd0322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:50.069308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23edc0-d8f5-4329-b922-fa8704fd0322 admin admin] PATH_INFO: `/v3/projects/caa81215b6274fb68edc6e3e57419da9/users/0e8dbb33c0494f01b5066affa7aa74c2/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:50.081272 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 959/1946] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:53:50.085437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ce86f-8cab-4d8a-ade7-b236b9af90d2 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:50.087829 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23edc0-d8f5-4329-b922-fa8704fd0322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caa81215b6274fb68edc6e3e57419da9, user_id=0e8dbb33c0494f01b5066affa7aa74c2, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:50.094870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23edc0-d8f5-4329-b922-fa8704fd0322 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:50.095399 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ce86f-8cab-4d8a-ade7-b236b9af90d2 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:50.100431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed55af6-7f75-4e41-ab8f-6aa485b4d0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:50.101083 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed55af6-7f75-4e41-ab8f-6aa485b4d0e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:50.101332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed55af6-7f75-4e41-ab8f-6aa485b4d0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:50.101653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed55af6-7f75-4e41-ab8f-6aa485b4d0e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:50.102417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed55af6-7f75-4e41-ab8f-6aa485b4d0e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:50.104378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed55af6-7f75-4e41-ab8f-6aa485b4d0e4 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:50.111074 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 988/1947] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:50 2022] PUT /identity/v3/projects/caa81215b6274fb68edc6e3e57419da9/users/0e8dbb33c0494f01b5066affa7aa74c2/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:53:50.116618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c929c-6e78-4fad-bee5-a51e9254caf1 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:50.116856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c929c-6e78-4fad-bee5-a51e9254caf1 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:50.118661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c929c-6e78-4fad-bee5-a51e9254caf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:50.138604 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-839c929c-6e78-4fad-bee5-a51e9254caf1 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:50.139812 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-839c929c-6e78-4fad-bee5-a51e9254caf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:50.141518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 960/1948] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:50 2022] POST /identity/v3/projects => generated 393 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:53:50.146027 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed55af6-7f75-4e41-ab8f-6aa485b4d0e4 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:50.183477 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:50.206538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 989/1949] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:53:50.215717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c929c-6e78-4fad-bee5-a51e9254caf1 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:50.217131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-839c929c-6e78-4fad-bee5-a51e9254caf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:50.217243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed55af6-7f75-4e41-ab8f-6aa485b4d0e4 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:50.222183 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca93e05-479c-4efa-85eb-7b9d46c860fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:50.222862 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca93e05-479c-4efa-85eb-7b9d46c860fc admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:50.223651 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca93e05-479c-4efa-85eb-7b9d46c860fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:50.223929 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca93e05-479c-4efa-85eb-7b9d46c860fc admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:50.224599 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca93e05-479c-4efa-85eb-7b9d46c860fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:50.226549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca93e05-479c-4efa-85eb-7b9d46c860fc admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:50.248881 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 961/1950] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:50 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:53:50.252853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca93e05-479c-4efa-85eb-7b9d46c860fc admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:50.263298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca93e05-479c-4efa-85eb-7b9d46c860fc admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:50.267796 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c1b69-ab36-4183-8637-09df444d9f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:50.268968 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c1b69-ab36-4183-8637-09df444d9f8b admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:50.269245 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c1b69-ab36-4183-8637-09df444d9f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:50.269515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c1b69-ab36-4183-8637-09df444d9f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:50.270195 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c1b69-ab36-4183-8637-09df444d9f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:50.271804 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c1b69-ab36-4183-8637-09df444d9f8b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:50.274681 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c929c-6e78-4fad-bee5-a51e9254caf1 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:50.278254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be5970-3754-4753-be9a-cb9db45c9357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:50.278898 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be5970-3754-4753-be9a-cb9db45c9357 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:50.279176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be5970-3754-4753-be9a-cb9db45c9357 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:50.279447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be5970-3754-4753-be9a-cb9db45c9357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:50.285372 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 962/1951] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:53:50.286538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c1b69-ab36-4183-8637-09df444d9f8b admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:50.289445 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be5970-3754-4753-be9a-cb9db45c9357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:50.290906 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be5970-3754-4753-be9a-cb9db45c9357 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:50.296024 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c1b69-ab36-4183-8637-09df444d9f8b admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:50.300311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 990/1952] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:53:50.301309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82867f33-018d-4504-8846-068acb6f0d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:50.302533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82867f33-018d-4504-8846-068acb6f0d3f admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:50.302533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82867f33-018d-4504-8846-068acb6f0d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:50.306080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82867f33-018d-4504-8846-068acb6f0d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:50.306080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82867f33-018d-4504-8846-068acb6f0d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:50.306080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82867f33-018d-4504-8846-068acb6f0d3f admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:50.320581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 963/1953] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:53:50.322015 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be5970-3754-4753-be9a-cb9db45c9357 service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:50.335728 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be5970-3754-4753-be9a-cb9db45c9357 service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:50.340419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29b989-9c13-4532-b9ce-7ca4f613fac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:50.341088 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29b989-9c13-4532-b9ce-7ca4f613fac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:50.341427 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29b989-9c13-4532-b9ce-7ca4f613fac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:50.341665 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29b989-9c13-4532-b9ce-7ca4f613fac5 admin admin] PATH_INFO: `/v3/projects/1a2854d9c830464c80979cdef896c0c5/users/9c0cb8d780634d949f322dc2bf9d7cef/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:50.359714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29b989-9c13-4532-b9ce-7ca4f613fac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a2854d9c830464c80979cdef896c0c5, user_id=9c0cb8d780634d949f322dc2bf9d7cef, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:50.362501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29b989-9c13-4532-b9ce-7ca4f613fac5 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:50.384123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 991/1954] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:50 2022] PUT /identity/v3/projects/1a2854d9c830464c80979cdef896c0c5/users/9c0cb8d780634d949f322dc2bf9d7cef/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:50.390213 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f02d07-e647-4e8b-b9be-e4ebbe71829e None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:50.390531 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f02d07-e647-4e8b-b9be-e4ebbe71829e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:50.390813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f02d07-e647-4e8b-b9be-e4ebbe71829e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:50.426413 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-59f02d07-e647-4e8b-b9be-e4ebbe71829e None None] MFA Rules not processed for user `9c0cb8d780634d949f322dc2bf9d7cef`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:50.428065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-59f02d07-e647-4e8b-b9be-e4ebbe71829e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:50.452890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:50.530747 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 964/1955] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:50.535612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29b989-9c13-4532-b9ce-7ca4f613fac5 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:50.604006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29b989-9c13-4532-b9ce-7ca4f613fac5 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:50.608720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbd950-94ac-4804-a68f-9be7cda25c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:50.609422 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbd950-94ac-4804-a68f-9be7cda25c77 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:50.609701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbd950-94ac-4804-a68f-9be7cda25c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:50.610031 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cbd950-94ac-4804-a68f-9be7cda25c77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:50.610670 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbd950-94ac-4804-a68f-9be7cda25c77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:50.612630 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cbd950-94ac-4804-a68f-9be7cda25c77 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:50.641227 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 992/1956] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:50 2022] POST /identity/v3/projects => generated 391 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:53:50.645971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f02d07-e647-4e8b-b9be-e4ebbe71829e None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:50.658079 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f02d07-e647-4e8b-b9be-e4ebbe71829e None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:50.739202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a983a5-cbaf-46bf-9852-868c48e43ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:50.739835 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a983a5-cbaf-46bf-9852-868c48e43ad3 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:50.740113 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a983a5-cbaf-46bf-9852-868c48e43ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:50.740385 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a983a5-cbaf-46bf-9852-868c48e43ad3 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:50.741067 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a983a5-cbaf-46bf-9852-868c48e43ad3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:50.743049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a983a5-cbaf-46bf-9852-868c48e43ad3 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:50.766216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 965/1957] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:50 2022] POST /identity/v3/users => generated 302 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:50.771081 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbd950-94ac-4804-a68f-9be7cda25c77 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:50.781739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cbd950-94ac-4804-a68f-9be7cda25c77 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:50.786713 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be1840-03a0-49ec-88b4-1dff9e5a34d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:50.787343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be1840-03a0-49ec-88b4-1dff9e5a34d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:50.787624 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be1840-03a0-49ec-88b4-1dff9e5a34d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:50.787896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be1840-03a0-49ec-88b4-1dff9e5a34d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:50.788599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be1840-03a0-49ec-88b4-1dff9e5a34d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:50.790228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be1840-03a0-49ec-88b4-1dff9e5a34d2 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:50.802900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 993/1958] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:50.807206 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a983a5-cbaf-46bf-9852-868c48e43ad3 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:50.818862 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a983a5-cbaf-46bf-9852-868c48e43ad3 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:50.824250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e83ce2-eaab-4f7b-a078-232201989365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:50.825356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e83ce2-eaab-4f7b-a078-232201989365 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:50.825356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e83ce2-eaab-4f7b-a078-232201989365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:50.825556 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e83ce2-eaab-4f7b-a078-232201989365 admin admin] PATH_INFO: `/v3/projects/6166b12f100f48a2b2495564c7430ebc/users/9d7587f3fead4382b4caf8f8a27a6be4/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:50.854559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e83ce2-eaab-4f7b-a078-232201989365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6166b12f100f48a2b2495564c7430ebc, user_id=9d7587f3fead4382b4caf8f8a27a6be4, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:50.858539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e83ce2-eaab-4f7b-a078-232201989365 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:50.892645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 966/1959] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:50 2022] PUT /identity/v3/projects/6166b12f100f48a2b2495564c7430ebc/users/9d7587f3fead4382b4caf8f8a27a6be4/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:53:50.898093 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be1840-03a0-49ec-88b4-1dff9e5a34d2 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:50.909431 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be1840-03a0-49ec-88b4-1dff9e5a34d2 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:50.954076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:50.973765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11ccc7d-aef2-4fb9-b835-67f7c705e7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:50.974401 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ccc7d-aef2-4fb9-b835-67f7c705e7ee admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:50.974698 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ccc7d-aef2-4fb9-b835-67f7c705e7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:50.974971 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11ccc7d-aef2-4fb9-b835-67f7c705e7ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:50.975690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ccc7d-aef2-4fb9-b835-67f7c705e7ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:50.977277 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11ccc7d-aef2-4fb9-b835-67f7c705e7ee admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:50.988818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 994/1960] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:50.993451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e83ce2-eaab-4f7b-a078-232201989365 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:51.005814 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e83ce2-eaab-4f7b-a078-232201989365 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:51.013541 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e9e37-9f42-4f9c-906b-7d6111a5351b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:51.014223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e9e37-9f42-4f9c-906b-7d6111a5351b admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:51.014640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e9e37-9f42-4f9c-906b-7d6111a5351b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:51.014823 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e9e37-9f42-4f9c-906b-7d6111a5351b admin admin] PATH_INFO: `/v3/projects/6166b12f100f48a2b2495564c7430ebc/users/9d7587f3fead4382b4caf8f8a27a6be4/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:51.017239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e9e37-9f42-4f9c-906b-7d6111a5351b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6166b12f100f48a2b2495564c7430ebc, user_id=9d7587f3fead4382b4caf8f8a27a6be4, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:51.020503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e9e37-9f42-4f9c-906b-7d6111a5351b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:51.032805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 967/1961] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:50 2022] PUT /identity/v3/projects/6166b12f100f48a2b2495564c7430ebc/users/9d7587f3fead4382b4caf8f8a27a6be4/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:51.043078 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231be18-bc63-43e2-8876-3361d9e3fb1e None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:51.043617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231be18-bc63-43e2-8876-3361d9e3fb1e None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:51.044064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231be18-bc63-43e2-8876-3361d9e3fb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:51.093354 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-d231be18-bc63-43e2-8876-3361d9e3fb1e None None] MFA Rules not processed for user `9d7587f3fead4382b4caf8f8a27a6be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:51.094709 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d231be18-bc63-43e2-8876-3361d9e3fb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:51.184118 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 995/1962] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:51 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:53:51.200311 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e9e37-9f42-4f9c-906b-7d6111a5351b admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:51.202179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-129e9e37-9f42-4f9c-906b-7d6111a5351b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:51.289078 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e9e37-9f42-4f9c-906b-7d6111a5351b admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:51.292861 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6f3af-849c-40b7-b8b5-d2be52d64706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:51.293503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6f3af-849c-40b7-b8b5-d2be52d64706 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:51.293784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6f3af-849c-40b7-b8b5-d2be52d64706 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:51.294056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6f3af-849c-40b7-b8b5-d2be52d64706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:51.322696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6f3af-849c-40b7-b8b5-d2be52d64706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:51.324488 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6f3af-849c-40b7-b8b5-d2be52d64706 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:51.336808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 968/1963] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:51 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:51.343164 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d18e34-ad3e-4a5f-8c6c-638afdecee4c None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:51.343455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d18e34-ad3e-4a5f-8c6c-638afdecee4c None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:51.343731 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d18e34-ad3e-4a5f-8c6c-638afdecee4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:51.387993 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-f3d18e34-ad3e-4a5f-8c6c-638afdecee4c None None] MFA Rules not processed for user `0e8dbb33c0494f01b5066affa7aa74c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:51.389673 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f3d18e34-ad3e-4a5f-8c6c-638afdecee4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:51.396231 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6f3af-849c-40b7-b8b5-d2be52d64706 service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:51.398197 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-18f6f3af-849c-40b7-b8b5-d2be52d64706 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:51.414456 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6f3af-849c-40b7-b8b5-d2be52d64706 service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:51.425650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc892e76-3aab-4dca-91ec-5545a5a33ff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:51.426735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc892e76-3aab-4dca-91ec-5545a5a33ff6 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:51.427088 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc892e76-3aab-4dca-91ec-5545a5a33ff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:51.427430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc892e76-3aab-4dca-91ec-5545a5a33ff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:51.438005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:51.486765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 996/1964] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:51.501454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d18e34-ad3e-4a5f-8c6c-638afdecee4c None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:51.511265 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc892e76-3aab-4dca-91ec-5545a5a33ff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:51.512663 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d18e34-ad3e-4a5f-8c6c-638afdecee4c None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:51.513153 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc892e76-3aab-4dca-91ec-5545a5a33ff6 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:51.516844 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89203e3f-c082-4aa5-a447-615b1b0786f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:51.517790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89203e3f-c082-4aa5-a447-615b1b0786f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:51.518398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89203e3f-c082-4aa5-a447-615b1b0786f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:51.519393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89203e3f-c082-4aa5-a447-615b1b0786f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:51.527989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 969/1965] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:53:51.529379 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89203e3f-c082-4aa5-a447-615b1b0786f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:51.530872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89203e3f-c082-4aa5-a447-615b1b0786f8 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:51.550171 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 997/1966] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:53:52.987699 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7094ac9-1c37-4afa-b245-861bd35d2d67 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:52.988259 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7094ac9-1c37-4afa-b245-861bd35d2d67 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:52.988542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7094ac9-1c37-4afa-b245-861bd35d2d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:53.012935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-d7094ac9-1c37-4afa-b245-861bd35d2d67 None None] MFA Rules not processed for user `0e8dbb33c0494f01b5066affa7aa74c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:53.014352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d7094ac9-1c37-4afa-b245-861bd35d2d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:53.032742 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 970/1967] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:54.213220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3bec1-e4dc-4fda-9d96-1e4830916e62 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:54.214141 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3bec1-e4dc-4fda-9d96-1e4830916e62 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:54.214444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3bec1-e4dc-4fda-9d96-1e4830916e62 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:54.216546 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 998/1968] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 3 12:53:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:53:54.233860 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df2855-adbd-429a-82d8-ab501d409566 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:54.234819 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df2855-adbd-429a-82d8-ab501d409566 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:54.235681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df2855-adbd-429a-82d8-ab501d409566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:54.288327 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a7df2855-adbd-429a-82d8-ab501d409566 None None] MFA Rules not processed for user `7b634f567dca42dc95df175a4f36c1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:54.289619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a7df2855-adbd-429a-82d8-ab501d409566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:54.404293 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 971/1969] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 3 12:53:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:54.461261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3bec1-e4dc-4fda-9d96-1e4830916e62 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:54.464613 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-21f3bec1-e4dc-4fda-9d96-1e4830916e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:54.504025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3bec1-e4dc-4fda-9d96-1e4830916e62 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:54.521918 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b08680-11ce-4b18-8c14-e62725916d5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:54.522937 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b08680-11ce-4b18-8c14-e62725916d5a service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:54.523701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b08680-11ce-4b18-8c14-e62725916d5a service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:54.524761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b08680-11ce-4b18-8c14-e62725916d5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:54.540301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b08680-11ce-4b18-8c14-e62725916d5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:54.542134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b08680-11ce-4b18-8c14-e62725916d5a service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:54.569982 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 999/1970] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:55.234309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df2855-adbd-429a-82d8-ab501d409566 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:55.307539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df2855-adbd-429a-82d8-ab501d409566 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:55.312087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f449ef4-969d-4fb6-a0ae-5dc211e67b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:55.312731 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f449ef4-969d-4fb6-a0ae-5dc211e67b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:55.313013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f449ef4-969d-4fb6-a0ae-5dc211e67b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:55.313283 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f449ef4-969d-4fb6-a0ae-5dc211e67b07 admin admin] PATH_INFO: `/v3/users/9c0cb8d780634d949f322dc2bf9d7cef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:55.314687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f449ef4-969d-4fb6-a0ae-5dc211e67b07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c0cb8d780634d949f322dc2bf9d7cef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:55.316686 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f449ef4-969d-4fb6-a0ae-5dc211e67b07 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:55.357975 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:55.364588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-7f449ef4-969d-4fb6-a0ae-5dc211e67b07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c0cb8d780634d949f322dc2bf9d7cef'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:55.376525 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-7f449ef4-969d-4fb6-a0ae-5dc211e67b07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c0cb8d780634d949f322dc2bf9d7cef'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:55.383589 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-7f449ef4-969d-4fb6-a0ae-5dc211e67b07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c0cb8d780634d949f322dc2bf9d7cef'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:55.384185 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-7f449ef4-969d-4fb6-a0ae-5dc211e67b07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c0cb8d780634d949f322dc2bf9d7cef'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:55.390863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-7f449ef4-969d-4fb6-a0ae-5dc211e67b07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c0cb8d780634d949f322dc2bf9d7cef'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:55.397202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 972/1971] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:55 2022] DELETE /identity/v3/users/9c0cb8d780634d949f322dc2bf9d7cef => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:55.401867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911bd29-9dbe-42ba-816c-3a873a0e8dd6 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:55.402396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911bd29-9dbe-42ba-816c-3a873a0e8dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:55.402518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911bd29-9dbe-42ba-816c-3a873a0e8dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:55.422273 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-c911bd29-9dbe-42ba-816c-3a873a0e8dd6 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:55.423478 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c911bd29-9dbe-42ba-816c-3a873a0e8dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:55.441883 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:55.484258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1000/1972] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:55.494273 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f449ef4-969d-4fb6-a0ae-5dc211e67b07 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:55.495679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-7f449ef4-969d-4fb6-a0ae-5dc211e67b07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:55.552032 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f449ef4-969d-4fb6-a0ae-5dc211e67b07 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:55.555706 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52412127-030a-4a96-b803-c80a457ae51c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:55.556320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52412127-030a-4a96-b803-c80a457ae51c service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:55.556602 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52412127-030a-4a96-b803-c80a457ae51c service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:55.556875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52412127-030a-4a96-b803-c80a457ae51c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:55.565843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52412127-030a-4a96-b803-c80a457ae51c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:55.567285 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52412127-030a-4a96-b803-c80a457ae51c service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:55.576576 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 973/1973] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:55.847182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911bd29-9dbe-42ba-816c-3a873a0e8dd6 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:55.848872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c911bd29-9dbe-42ba-816c-3a873a0e8dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:55.861060 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911bd29-9dbe-42ba-816c-3a873a0e8dd6 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:55.865919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15090f2-f658-48d3-80f2-7325d1b179dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:55.871090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15090f2-f658-48d3-80f2-7325d1b179dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:55.871422 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15090f2-f658-48d3-80f2-7325d1b179dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:55.871645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15090f2-f658-48d3-80f2-7325d1b179dc admin admin] PATH_INFO: `/v3/projects/1a2854d9c830464c80979cdef896c0c5` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:55.872933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15090f2-f658-48d3-80f2-7325d1b179dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a2854d9c830464c80979cdef896c0c5)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:55.874946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15090f2-f658-48d3-80f2-7325d1b179dc admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:55.921528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d15090f2-f658-48d3-80f2-7325d1b179dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a2854d9c830464c80979cdef896c0c5', 'request_id': 'req-d15090f2-f658-48d3-80f2-7325d1b179dc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:55.928265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d15090f2-f658-48d3-80f2-7325d1b179dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a2854d9c830464c80979cdef896c0c5', 'request_id': 'req-d15090f2-f658-48d3-80f2-7325d1b179dc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:55.934234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d15090f2-f658-48d3-80f2-7325d1b179dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a2854d9c830464c80979cdef896c0c5', 'request_id': 'req-d15090f2-f658-48d3-80f2-7325d1b179dc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:55.946969 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d15090f2-f658-48d3-80f2-7325d1b179dc admin admin] The token cache is being invalidate because project 1a2854d9c830464c80979cdef896c0c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:55.947268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d15090f2-f658-48d3-80f2-7325d1b179dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:55.947927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d15090f2-f658-48d3-80f2-7325d1b179dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:55.949464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1001/1974] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:55 2022] DELETE /identity/v3/projects/1a2854d9c830464c80979cdef896c0c5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:55.953581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52412127-030a-4a96-b803-c80a457ae51c service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:55.964687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52412127-030a-4a96-b803-c80a457ae51c service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:56.021476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76b66c-d2bc-4db1-848e-e21eff8b8d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:56.022112 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76b66c-d2bc-4db1-848e-e21eff8b8d63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:56.022388 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76b66c-d2bc-4db1-848e-e21eff8b8d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:56.022702 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76b66c-d2bc-4db1-848e-e21eff8b8d63 admin admin] PATH_INFO: `/v3/users/9d7587f3fead4382b4caf8f8a27a6be4` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:56.023893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76b66c-d2bc-4db1-848e-e21eff8b8d63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d7587f3fead4382b4caf8f8a27a6be4)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:56.025814 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76b66c-d2bc-4db1-848e-e21eff8b8d63 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:56.079858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8f76b66c-d2bc-4db1-848e-e21eff8b8d63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d7587f3fead4382b4caf8f8a27a6be4'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:56.090379 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:56.093797 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8f76b66c-d2bc-4db1-848e-e21eff8b8d63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d7587f3fead4382b4caf8f8a27a6be4'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:56.102455 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8f76b66c-d2bc-4db1-848e-e21eff8b8d63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d7587f3fead4382b4caf8f8a27a6be4'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:56.103185 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8f76b66c-d2bc-4db1-848e-e21eff8b8d63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d7587f3fead4382b4caf8f8a27a6be4'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:56.110507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8f76b66c-d2bc-4db1-848e-e21eff8b8d63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d7587f3fead4382b4caf8f8a27a6be4'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:56.117132 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 974/1975] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:53:55 2022] DELETE /identity/v3/users/9d7587f3fead4382b4caf8f8a27a6be4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:56.525793 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15090f2-f658-48d3-80f2-7325d1b179dc admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:56.527425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d15090f2-f658-48d3-80f2-7325d1b179dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:56.553592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15090f2-f658-48d3-80f2-7325d1b179dc admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:56.613338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a2d3b-8c57-4dd1-bc7c-92d0e14b8a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:56.614041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a2d3b-8c57-4dd1-bc7c-92d0e14b8a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:56.614317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a2d3b-8c57-4dd1-bc7c-92d0e14b8a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:56.614618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a2d3b-8c57-4dd1-bc7c-92d0e14b8a1d admin admin] PATH_INFO: `/v3/projects/6166b12f100f48a2b2495564c7430ebc` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:56.615792 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a2d3b-8c57-4dd1-bc7c-92d0e14b8a1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6166b12f100f48a2b2495564c7430ebc)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:56.617669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a2d3b-8c57-4dd1-bc7c-92d0e14b8a1d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:56.665981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-cb5a2d3b-8c57-4dd1-bc7c-92d0e14b8a1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6166b12f100f48a2b2495564c7430ebc', 'request_id': 'req-cb5a2d3b-8c57-4dd1-bc7c-92d0e14b8a1d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:56.667664 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:56.676281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-cb5a2d3b-8c57-4dd1-bc7c-92d0e14b8a1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6166b12f100f48a2b2495564c7430ebc', 'request_id': 'req-cb5a2d3b-8c57-4dd1-bc7c-92d0e14b8a1d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:56.683678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-cb5a2d3b-8c57-4dd1-bc7c-92d0e14b8a1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6166b12f100f48a2b2495564c7430ebc', 'request_id': 'req-cb5a2d3b-8c57-4dd1-bc7c-92d0e14b8a1d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:56.695968 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-cb5a2d3b-8c57-4dd1-bc7c-92d0e14b8a1d admin admin] The token cache is being invalidate because project 6166b12f100f48a2b2495564c7430ebc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:53:56.696452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-cb5a2d3b-8c57-4dd1-bc7c-92d0e14b8a1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:56.697114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-cb5a2d3b-8c57-4dd1-bc7c-92d0e14b8a1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:53:56.698772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1002/1976] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:53:56 2022] DELETE /identity/v3/projects/6166b12f100f48a2b2495564c7430ebc => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:56.703791 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b577338-20d4-4fd5-8101-c886e5559e32 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:56.704285 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b577338-20d4-4fd5-8101-c886e5559e32 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:56.704380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b577338-20d4-4fd5-8101-c886e5559e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:56.725280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-8b577338-20d4-4fd5-8101-c886e5559e32 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:56.726500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8b577338-20d4-4fd5-8101-c886e5559e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:56.784573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 975/1977] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:56.793190 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4be681-59c6-432b-a969-bc7ae6f1f85f None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:56.793506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4be681-59c6-432b-a969-bc7ae6f1f85f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:56.793794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4be681-59c6-432b-a969-bc7ae6f1f85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:56.814238 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-7a4be681-59c6-432b-a969-bc7ae6f1f85f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:56.816139 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-7a4be681-59c6-432b-a969-bc7ae6f1f85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:56.825255 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1003/1978] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:56.830170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b577338-20d4-4fd5-8101-c886e5559e32 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:56.840280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b577338-20d4-4fd5-8101-c886e5559e32 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:56.845014 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8286c-d4fd-458e-b9d9-31c70b69f61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:56.845631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8286c-d4fd-458e-b9d9-31c70b69f61c admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:56.845909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8286c-d4fd-458e-b9d9-31c70b69f61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:56.846181 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8286c-d4fd-458e-b9d9-31c70b69f61c admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:56.846909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8286c-d4fd-458e-b9d9-31c70b69f61c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:53:56.847211 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8286c-d4fd-458e-b9d9-31c70b69f61c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:56.848819 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8286c-d4fd-458e-b9d9-31c70b69f61c admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:56.860543 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 976/1979] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:53:56.865426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4be681-59c6-432b-a969-bc7ae6f1f85f None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:56.874739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4be681-59c6-432b-a969-bc7ae6f1f85f None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:56.880661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe574b8-ec0c-4084-a355-d5920fd4cbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:56.880661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe574b8-ec0c-4084-a355-d5920fd4cbab admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:56.880962 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe574b8-ec0c-4084-a355-d5920fd4cbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:56.881128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe574b8-ec0c-4084-a355-d5920fd4cbab admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:56.881814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe574b8-ec0c-4084-a355-d5920fd4cbab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:56.883777 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe574b8-ec0c-4084-a355-d5920fd4cbab admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:56.911965 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1004/1980] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:56 2022] POST /identity/v3/projects => generated 407 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:56.916312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8286c-d4fd-458e-b9d9-31c70b69f61c admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:56.926973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8286c-d4fd-458e-b9d9-31c70b69f61c admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:56.981521 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9825a22-f1b0-40bf-8c7b-18f886cc042a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:56.982122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9825a22-f1b0-40bf-8c7b-18f886cc042a admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:56.982401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9825a22-f1b0-40bf-8c7b-18f886cc042a admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:56.982701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9825a22-f1b0-40bf-8c7b-18f886cc042a admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:56.983396 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9825a22-f1b0-40bf-8c7b-18f886cc042a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:56.985250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9825a22-f1b0-40bf-8c7b-18f886cc042a admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:57.012348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 977/1981] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:56 2022] POST /identity/v3/users => generated 304 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:57.013527 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe574b8-ec0c-4084-a355-d5920fd4cbab admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:57.023871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe574b8-ec0c-4084-a355-d5920fd4cbab admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:57.028436 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca22b4-d511-407d-8316-2a41be9dca97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:57.029047 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca22b4-d511-407d-8316-2a41be9dca97 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:57.029329 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca22b4-d511-407d-8316-2a41be9dca97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:57.029604 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca22b4-d511-407d-8316-2a41be9dca97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:57.030297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca22b4-d511-407d-8316-2a41be9dca97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:57.031936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca22b4-d511-407d-8316-2a41be9dca97 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:57.041733 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1005/1982] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:53:57.045463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9825a22-f1b0-40bf-8c7b-18f886cc042a admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:57.056165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9825a22-f1b0-40bf-8c7b-18f886cc042a admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:57.060693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd98bd-3852-4d95-9785-e09e368d9c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:57.061289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd98bd-3852-4d95-9785-e09e368d9c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:57.061567 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd98bd-3852-4d95-9785-e09e368d9c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:57.061838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd98bd-3852-4d95-9785-e09e368d9c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:57.062567 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd98bd-3852-4d95-9785-e09e368d9c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:57.064671 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd98bd-3852-4d95-9785-e09e368d9c69 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:57.075751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 978/1983] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:57.079903 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca22b4-d511-407d-8316-2a41be9dca97 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:57.090020 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca22b4-d511-407d-8316-2a41be9dca97 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:57.094673 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c70a7-fdc6-4716-8736-f22d91f2c0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:57.095314 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c70a7-fdc6-4716-8736-f22d91f2c0e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:57.095594 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c70a7-fdc6-4716-8736-f22d91f2c0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:57.095874 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c70a7-fdc6-4716-8736-f22d91f2c0e4 admin admin] PATH_INFO: `/v3/projects/20a5b0b324374f3da70074e32975d3d7/users/b2347459a1d94c8096247a879aa126f9/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:57.109732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c70a7-fdc6-4716-8736-f22d91f2c0e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a5b0b324374f3da70074e32975d3d7, user_id=b2347459a1d94c8096247a879aa126f9, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:57.112857 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c70a7-fdc6-4716-8736-f22d91f2c0e4 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:57.125335 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1006/1984] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:57 2022] PUT /identity/v3/projects/20a5b0b324374f3da70074e32975d3d7/users/b2347459a1d94c8096247a879aa126f9/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:53:57.135890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a8f65-0c70-46bc-9807-58710b3e4a84 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:57.136180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a8f65-0c70-46bc-9807-58710b3e4a84 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:57.136459 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a8f65-0c70-46bc-9807-58710b3e4a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:57.163829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-155a8f65-0c70-46bc-9807-58710b3e4a84 None None] MFA Rules not processed for user `b2347459a1d94c8096247a879aa126f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:57.165411 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-155a8f65-0c70-46bc-9807-58710b3e4a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:57.198842 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:57.234078 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 979/1985] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:57 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:57.239356 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c70a7-fdc6-4716-8736-f22d91f2c0e4 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:57.305059 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c70a7-fdc6-4716-8736-f22d91f2c0e4 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:57.309775 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334dfa51-579a-42b6-aed3-6de2acdf407b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:57.310425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334dfa51-579a-42b6-aed3-6de2acdf407b admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:57.310745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334dfa51-579a-42b6-aed3-6de2acdf407b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:57.311022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334dfa51-579a-42b6-aed3-6de2acdf407b admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:57.311691 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334dfa51-579a-42b6-aed3-6de2acdf407b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:57.313615 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334dfa51-579a-42b6-aed3-6de2acdf407b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:57.340879 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1007/1986] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:53:57 2022] POST /identity/v3/projects => generated 407 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:57.345245 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a8f65-0c70-46bc-9807-58710b3e4a84 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:57.356777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a8f65-0c70-46bc-9807-58710b3e4a84 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:57.430279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52b8f4-692d-4236-a151-a43b162ad8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:57.430946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52b8f4-692d-4236-a151-a43b162ad8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:57.431229 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52b8f4-692d-4236-a151-a43b162ad8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:57.432126 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52b8f4-692d-4236-a151-a43b162ad8c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:57.434756 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52b8f4-692d-4236-a151-a43b162ad8c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:57.434756 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a52b8f4-692d-4236-a151-a43b162ad8c4 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:57.458574 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 980/1987] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:53:57 2022] POST /identity/v3/users => generated 310 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:53:57.462220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334dfa51-579a-42b6-aed3-6de2acdf407b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:57.475761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334dfa51-579a-42b6-aed3-6de2acdf407b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:57.482379 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68055cc2-cfe6-4e54-95c6-0c231c67b4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:57.483081 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68055cc2-cfe6-4e54-95c6-0c231c67b4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:57.483506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68055cc2-cfe6-4e54-95c6-0c231c67b4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:57.483788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68055cc2-cfe6-4e54-95c6-0c231c67b4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:57.484495 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68055cc2-cfe6-4e54-95c6-0c231c67b4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:57.486103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68055cc2-cfe6-4e54-95c6-0c231c67b4d3 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:57.498524 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1008/1988] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:57.503832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52b8f4-692d-4236-a151-a43b162ad8c4 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:57.515687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a52b8f4-692d-4236-a151-a43b162ad8c4 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:57.521040 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbcb7c-b764-4bfd-bd6f-ce7ac9a2a2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:57.521723 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbcb7c-b764-4bfd-bd6f-ce7ac9a2a2c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:57.522007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbcb7c-b764-4bfd-bd6f-ce7ac9a2a2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:57.522284 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbcb7c-b764-4bfd-bd6f-ce7ac9a2a2c5 admin admin] PATH_INFO: `/v3/projects/944f2c5db12242bf95599e27efd7ce26/users/ae72714f29b843caa9f7987362f62bd1/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:57.539047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbcb7c-b764-4bfd-bd6f-ce7ac9a2a2c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=944f2c5db12242bf95599e27efd7ce26, user_id=ae72714f29b843caa9f7987362f62bd1, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:57.541970 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccbcb7c-b764-4bfd-bd6f-ce7ac9a2a2c5 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:57.573020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 981/1989] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:57 2022] PUT /identity/v3/projects/944f2c5db12242bf95599e27efd7ce26/users/ae72714f29b843caa9f7987362f62bd1/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:57.579073 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68055cc2-cfe6-4e54-95c6-0c231c67b4d3 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:57.592838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68055cc2-cfe6-4e54-95c6-0c231c67b4d3 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:57.664944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:53:57.677322 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509d27a-4c4e-46a5-9e4c-b196ba47d83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:57.677951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509d27a-4c4e-46a5-9e4c-b196ba47d83e admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:57.678242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509d27a-4c4e-46a5-9e4c-b196ba47d83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:57.679483 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509d27a-4c4e-46a5-9e4c-b196ba47d83e admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:57.679653 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509d27a-4c4e-46a5-9e4c-b196ba47d83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:57.681517 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509d27a-4c4e-46a5-9e4c-b196ba47d83e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:57.696815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1009/1990] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:53:57.700979 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbcb7c-b764-4bfd-bd6f-ce7ac9a2a2c5 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:57.716077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbcb7c-b764-4bfd-bd6f-ce7ac9a2a2c5 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:57.744580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fc5da-2276-4571-9953-b550100976eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:57.745273 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fc5da-2276-4571-9953-b550100976eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:57.745546 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fc5da-2276-4571-9953-b550100976eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:57.745941 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fc5da-2276-4571-9953-b550100976eb admin admin] PATH_INFO: `/v3/projects/944f2c5db12242bf95599e27efd7ce26/users/ae72714f29b843caa9f7987362f62bd1/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:57.748308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972fc5da-2276-4571-9953-b550100976eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=944f2c5db12242bf95599e27efd7ce26, user_id=ae72714f29b843caa9f7987362f62bd1, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:57.750619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972fc5da-2276-4571-9953-b550100976eb admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:57.762172 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 982/1991] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:53:57 2022] PUT /identity/v3/projects/944f2c5db12242bf95599e27efd7ce26/users/ae72714f29b843caa9f7987362f62bd1/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:53:57.773878 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d1552-e53c-4785-8f66-9be0cc0b7499 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:57.774169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d1552-e53c-4785-8f66-9be0cc0b7499 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:57.774562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d1552-e53c-4785-8f66-9be0cc0b7499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:57.806767 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-db9d1552-e53c-4785-8f66-9be0cc0b7499 None None] MFA Rules not processed for user `ae72714f29b843caa9f7987362f62bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:53:57.807992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-db9d1552-e53c-4785-8f66-9be0cc0b7499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:57.910394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1010/1992] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:53:57 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:53:57.936946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fc5da-2276-4571-9953-b550100976eb admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:57.939670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-972fc5da-2276-4571-9953-b550100976eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:53:58.017928 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fc5da-2276-4571-9953-b550100976eb admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:58.030400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2ae03-c49d-45e1-ad8b-21334b54eaab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:58.031113 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2ae03-c49d-45e1-ad8b-21334b54eaab service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:58.034512 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2ae03-c49d-45e1-ad8b-21334b54eaab service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:58.034877 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea2ae03-c49d-45e1-ad8b-21334b54eaab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:58.065760 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2ae03-c49d-45e1-ad8b-21334b54eaab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:58.067888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea2ae03-c49d-45e1-ad8b-21334b54eaab service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:58.084383 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 983/1993] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:57 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:53:58.322770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d1552-e53c-4785-8f66-9be0cc0b7499 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:53:58.333114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d1552-e53c-4785-8f66-9be0cc0b7499 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:53:58.336741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e3079-28a4-4bba-b721-d4ef532dd481 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:53:58.337395 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e3079-28a4-4bba-b721-d4ef532dd481 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:53:58.337676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e3079-28a4-4bba-b721-d4ef532dd481 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:53:58.337948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e3079-28a4-4bba-b721-d4ef532dd481 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:53:58.403978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e3079-28a4-4bba-b721-d4ef532dd481 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:53:58.411219 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e3079-28a4-4bba-b721-d4ef532dd481 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:53:58.436420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1011/1994] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:53:58 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:05.980071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2ae03-c49d-45e1-ad8b-21334b54eaab service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:05.981541 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-fea2ae03-c49d-45e1-ad8b-21334b54eaab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:06.063325 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:06.133389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea2ae03-c49d-45e1-ad8b-21334b54eaab service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:06.147542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802cdd1-bc14-4ea5-aded-379b410fca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:06.148210 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802cdd1-bc14-4ea5-aded-379b410fca87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:06.148494 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802cdd1-bc14-4ea5-aded-379b410fca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:06.148771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802cdd1-bc14-4ea5-aded-379b410fca87 admin admin] PATH_INFO: `/v3/users/d7bf4677b7c04cd687673f01de256ff1` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:06.150060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802cdd1-bc14-4ea5-aded-379b410fca87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7bf4677b7c04cd687673f01de256ff1)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:06.152025 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802cdd1-bc14-4ea5-aded-379b410fca87 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:06.202267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0802cdd1-bc14-4ea5-aded-379b410fca87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7bf4677b7c04cd687673f01de256ff1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:06.216019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0802cdd1-bc14-4ea5-aded-379b410fca87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7bf4677b7c04cd687673f01de256ff1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:06.223721 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0802cdd1-bc14-4ea5-aded-379b410fca87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7bf4677b7c04cd687673f01de256ff1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:06.224474 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0802cdd1-bc14-4ea5-aded-379b410fca87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7bf4677b7c04cd687673f01de256ff1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:06.232439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0802cdd1-bc14-4ea5-aded-379b410fca87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7bf4677b7c04cd687673f01de256ff1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:06.238624 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 984/1995] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:05 2022] DELETE /identity/v3/users/d7bf4677b7c04cd687673f01de256ff1 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:06.243449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85a482-0330-4d81-8988-26c8ed7cac2d None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:06.244055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85a482-0330-4d81-8988-26c8ed7cac2d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:06.244055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85a482-0330-4d81-8988-26c8ed7cac2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:06.263576 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-fb85a482-0330-4d81-8988-26c8ed7cac2d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:06.264822 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-fb85a482-0330-4d81-8988-26c8ed7cac2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:06.328100 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1012/1996] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:06.338849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802cdd1-bc14-4ea5-aded-379b410fca87 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:06.340340 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-0802cdd1-bc14-4ea5-aded-379b410fca87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:06.398871 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802cdd1-bc14-4ea5-aded-379b410fca87 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:06.402859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103ff2b-2bd0-49cb-a4b9-02e7757fccef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:06.403538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103ff2b-2bd0-49cb-a4b9-02e7757fccef service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:06.403818 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103ff2b-2bd0-49cb-a4b9-02e7757fccef service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:06.404089 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103ff2b-2bd0-49cb-a4b9-02e7757fccef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:06.413939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103ff2b-2bd0-49cb-a4b9-02e7757fccef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:06.415385 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6103ff2b-2bd0-49cb-a4b9-02e7757fccef service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:06.424972 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 985/1997] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:06.767750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85a482-0330-4d81-8988-26c8ed7cac2d None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:06.769865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-fb85a482-0330-4d81-8988-26c8ed7cac2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:06.786188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85a482-0330-4d81-8988-26c8ed7cac2d None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:06.791022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112b434-e1f3-4a3b-b401-23f53d175f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:06.791675 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112b434-e1f3-4a3b-b401-23f53d175f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:06.791955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112b434-e1f3-4a3b-b401-23f53d175f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:06.792223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112b434-e1f3-4a3b-b401-23f53d175f98 admin admin] PATH_INFO: `/v3/projects/4ae6edf63ed64a6fa7fb3ee8b443f253` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:06.793388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112b434-e1f3-4a3b-b401-23f53d175f98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ae6edf63ed64a6fa7fb3ee8b443f253)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:06.795390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112b434-e1f3-4a3b-b401-23f53d175f98 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:06.798162 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:06.862797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-8112b434-e1f3-4a3b-b401-23f53d175f98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ae6edf63ed64a6fa7fb3ee8b443f253', 'request_id': 'req-8112b434-e1f3-4a3b-b401-23f53d175f98'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:06.873286 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-8112b434-e1f3-4a3b-b401-23f53d175f98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ae6edf63ed64a6fa7fb3ee8b443f253', 'request_id': 'req-8112b434-e1f3-4a3b-b401-23f53d175f98'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:06.880521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-8112b434-e1f3-4a3b-b401-23f53d175f98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ae6edf63ed64a6fa7fb3ee8b443f253', 'request_id': 'req-8112b434-e1f3-4a3b-b401-23f53d175f98'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:06.891961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-8112b434-e1f3-4a3b-b401-23f53d175f98 admin admin] The token cache is being invalidate because project 4ae6edf63ed64a6fa7fb3ee8b443f253 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:06.892250 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-8112b434-e1f3-4a3b-b401-23f53d175f98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:06.893409 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-8112b434-e1f3-4a3b-b401-23f53d175f98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:06.897869 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1013/1998] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:06 2022] DELETE /identity/v3/projects/4ae6edf63ed64a6fa7fb3ee8b443f253 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:06.899241 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103ff2b-2bd0-49cb-a4b9-02e7757fccef service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:06.911250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6103ff2b-2bd0-49cb-a4b9-02e7757fccef service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:06.964116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:06.982258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ed0244-9574-4466-9fab-a74123adea7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:06.985086 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed0244-9574-4466-9fab-a74123adea7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:06.985711 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed0244-9574-4466-9fab-a74123adea7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:06.986556 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ed0244-9574-4466-9fab-a74123adea7c admin admin] PATH_INFO: `/v3/users/86d221b3b09b4a51b99b3c9eb7c92ef5` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:06.988741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed0244-9574-4466-9fab-a74123adea7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=86d221b3b09b4a51b99b3c9eb7c92ef5)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:06.991807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ed0244-9574-4466-9fab-a74123adea7c admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:07.071335 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-43ed0244-9574-4466-9fab-a74123adea7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86d221b3b09b4a51b99b3c9eb7c92ef5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:07.089717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-43ed0244-9574-4466-9fab-a74123adea7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86d221b3b09b4a51b99b3c9eb7c92ef5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:07.099503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-43ed0244-9574-4466-9fab-a74123adea7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86d221b3b09b4a51b99b3c9eb7c92ef5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:07.099894 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-43ed0244-9574-4466-9fab-a74123adea7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86d221b3b09b4a51b99b3c9eb7c92ef5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:07.109217 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-43ed0244-9574-4466-9fab-a74123adea7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86d221b3b09b4a51b99b3c9eb7c92ef5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:07.115999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 986/1999] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:06 2022] DELETE /identity/v3/users/86d221b3b09b4a51b99b3c9eb7c92ef5 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:07.245051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112b434-e1f3-4a3b-b401-23f53d175f98 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:07.246587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8112b434-e1f3-4a3b-b401-23f53d175f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:07.269470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112b434-e1f3-4a3b-b401-23f53d175f98 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:07.350026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b63246-4361-486b-a666-c5f616e1421d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:07.350400 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b63246-4361-486b-a666-c5f616e1421d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:07.350711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b63246-4361-486b-a666-c5f616e1421d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:07.350995 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b63246-4361-486b-a666-c5f616e1421d admin admin] PATH_INFO: `/v3/users/b2347459a1d94c8096247a879aa126f9` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:07.352349 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b63246-4361-486b-a666-c5f616e1421d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2347459a1d94c8096247a879aa126f9)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:07.354307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b63246-4361-486b-a666-c5f616e1421d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:07.431070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e5b63246-4361-486b-a666-c5f616e1421d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2347459a1d94c8096247a879aa126f9'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:07.445369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e5b63246-4361-486b-a666-c5f616e1421d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2347459a1d94c8096247a879aa126f9'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:07.469895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e5b63246-4361-486b-a666-c5f616e1421d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2347459a1d94c8096247a879aa126f9'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:07.471141 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e5b63246-4361-486b-a666-c5f616e1421d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2347459a1d94c8096247a879aa126f9'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:07.472502 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e5b63246-4361-486b-a666-c5f616e1421d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2347459a1d94c8096247a879aa126f9'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:07.505400 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1014/2000] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:07 2022] DELETE /identity/v3/users/b2347459a1d94c8096247a879aa126f9 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:07.509802 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5428ed-03f6-484e-8092-5c8c03994156 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:07.510300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5428ed-03f6-484e-8092-5c8c03994156 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:07.510775 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5428ed-03f6-484e-8092-5c8c03994156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:07.544380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-8d5428ed-03f6-484e-8092-5c8c03994156 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:07.545766 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8d5428ed-03f6-484e-8092-5c8c03994156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:07.569181 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b63246-4361-486b-a666-c5f616e1421d admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:07.571207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e5b63246-4361-486b-a666-c5f616e1421d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:07.600419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:07.652887 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 987/2001] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:07.664893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5428ed-03f6-484e-8092-5c8c03994156 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:07.665634 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b63246-4361-486b-a666-c5f616e1421d admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:07.666943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8d5428ed-03f6-484e-8092-5c8c03994156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:07.670800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced6fb5d-73d8-4122-8ebc-cc7a64292388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:07.672467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6fb5d-73d8-4122-8ebc-cc7a64292388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:07.672761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6fb5d-73d8-4122-8ebc-cc7a64292388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:07.673098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced6fb5d-73d8-4122-8ebc-cc7a64292388 admin admin] PATH_INFO: `/v3/projects/f97f3d22e6c84afe96c12a516901d5d7` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:07.675126 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6fb5d-73d8-4122-8ebc-cc7a64292388 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f97f3d22e6c84afe96c12a516901d5d7)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:07.677356 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced6fb5d-73d8-4122-8ebc-cc7a64292388 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:07.744425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-ced6fb5d-73d8-4122-8ebc-cc7a64292388 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f97f3d22e6c84afe96c12a516901d5d7', 'request_id': 'req-ced6fb5d-73d8-4122-8ebc-cc7a64292388'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:07.752331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-ced6fb5d-73d8-4122-8ebc-cc7a64292388 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f97f3d22e6c84afe96c12a516901d5d7', 'request_id': 'req-ced6fb5d-73d8-4122-8ebc-cc7a64292388'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:07.753925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5428ed-03f6-484e-8092-5c8c03994156 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:07.759660 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-ced6fb5d-73d8-4122-8ebc-cc7a64292388 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f97f3d22e6c84afe96c12a516901d5d7', 'request_id': 'req-ced6fb5d-73d8-4122-8ebc-cc7a64292388'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:07.771119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-ced6fb5d-73d8-4122-8ebc-cc7a64292388 admin admin] The token cache is being invalidate because project f97f3d22e6c84afe96c12a516901d5d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:07.771411 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-ced6fb5d-73d8-4122-8ebc-cc7a64292388 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:07.772561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-ced6fb5d-73d8-4122-8ebc-cc7a64292388 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:07.774740 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1015/2002] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:07 2022] DELETE /identity/v3/projects/f97f3d22e6c84afe96c12a516901d5d7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:07.779252 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee72fdc-cd4a-436f-9c72-23260acdfedf None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:07.779542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee72fdc-cd4a-436f-9c72-23260acdfedf None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:07.779822 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee72fdc-cd4a-436f-9c72-23260acdfedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:07.792918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:07.803865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-cee72fdc-cd4a-436f-9c72-23260acdfedf None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:07.805271 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-cee72fdc-cd4a-436f-9c72-23260acdfedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:07.823193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7855de-5ace-4a53-862a-4f09cae0e773 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:07.823816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7855de-5ace-4a53-862a-4f09cae0e773 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:07.824097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7855de-5ace-4a53-862a-4f09cae0e773 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:07.824373 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7855de-5ace-4a53-862a-4f09cae0e773 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:07.826154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2e7855de-5ace-4a53-862a-4f09cae0e773 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:07.886605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1016/2003] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:54:07.894023 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea383b-b7ff-4fc7-8a3b-2acc8645fea0 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:07.894309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea383b-b7ff-4fc7-8a3b-2acc8645fea0 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:07.894617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea383b-b7ff-4fc7-8a3b-2acc8645fea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:07.913915 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7855de-5ace-4a53-862a-4f09cae0e773 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:07.919540 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-91ea383b-b7ff-4fc7-8a3b-2acc8645fea0 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:07.920882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-91ea383b-b7ff-4fc7-8a3b-2acc8645fea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:07.922302 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7855de-5ace-4a53-862a-4f09cae0e773 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:07.932429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1017/2004] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:54:07.935338 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 988/2005] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:54:07.936305 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea383b-b7ff-4fc7-8a3b-2acc8645fea0 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:07.945870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea383b-b7ff-4fc7-8a3b-2acc8645fea0 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:07.950779 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f214bc8-b10c-41e4-826f-908f7630c090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:07.951568 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f214bc8-b10c-41e4-826f-908f7630c090 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:07.951922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f214bc8-b10c-41e4-826f-908f7630c090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:07.952271 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f214bc8-b10c-41e4-826f-908f7630c090 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:07.953050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f214bc8-b10c-41e4-826f-908f7630c090 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:54:07.953443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f214bc8-b10c-41e4-826f-908f7630c090 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:07.955167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f214bc8-b10c-41e4-826f-908f7630c090 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:07.970691 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1018/2006] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:54:07.972131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7855de-5ace-4a53-862a-4f09cae0e773 service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:07.982949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7855de-5ace-4a53-862a-4f09cae0e773 service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:08.010785 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6abcf7-9d7f-4541-9931-5098219d2566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:08.011441 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6abcf7-9d7f-4541-9931-5098219d2566 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:08.011724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6abcf7-9d7f-4541-9931-5098219d2566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:08.012005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6abcf7-9d7f-4541-9931-5098219d2566 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:08.018109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6abcf7-9d7f-4541-9931-5098219d2566 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:08.020381 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6abcf7-9d7f-4541-9931-5098219d2566 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:08.073492 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 989/2007] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:07 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:54:08.078342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f214bc8-b10c-41e4-826f-908f7630c090 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:08.092466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f214bc8-b10c-41e4-826f-908f7630c090 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:08.134174 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:08.206515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193b08f-1f97-4c6b-8991-746514d9928b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:08.207245 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193b08f-1f97-4c6b-8991-746514d9928b admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:08.207599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193b08f-1f97-4c6b-8991-746514d9928b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:08.207944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193b08f-1f97-4c6b-8991-746514d9928b admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:08.215541 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193b08f-1f97-4c6b-8991-746514d9928b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:08.224627 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193b08f-1f97-4c6b-8991-746514d9928b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:08.261246 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1019/2008] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:08 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:54:08.264807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6abcf7-9d7f-4541-9931-5098219d2566 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:08.277168 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6abcf7-9d7f-4541-9931-5098219d2566 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:08.283298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825d193-554a-4f68-875a-e40f12808241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:08.284696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825d193-554a-4f68-875a-e40f12808241 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:08.284932 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825d193-554a-4f68-875a-e40f12808241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:08.285202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9825d193-554a-4f68-875a-e40f12808241 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:08.285954 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825d193-554a-4f68-875a-e40f12808241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:08.287653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9825d193-554a-4f68-875a-e40f12808241 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:08.302539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 990/2009] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:08.306419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193b08f-1f97-4c6b-8991-746514d9928b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:08.320778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193b08f-1f97-4c6b-8991-746514d9928b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:08.327355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bacc1-29e1-4002-bc3c-10eaf2b3327a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:08.328276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bacc1-29e1-4002-bc3c-10eaf2b3327a admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:08.328656 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bacc1-29e1-4002-bc3c-10eaf2b3327a admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:08.328829 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bacc1-29e1-4002-bc3c-10eaf2b3327a admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:08.329638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bacc1-29e1-4002-bc3c-10eaf2b3327a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:08.332245 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bacc1-29e1-4002-bc3c-10eaf2b3327a admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:08.349473 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1020/2010] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:08.350734 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825d193-554a-4f68-875a-e40f12808241 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:08.386546 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9825d193-554a-4f68-875a-e40f12808241 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:08.392770 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83c370-245a-4487-95b3-bac1cfeaa473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:08.394164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83c370-245a-4487-95b3-bac1cfeaa473 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:08.394326 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83c370-245a-4487-95b3-bac1cfeaa473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:08.394719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83c370-245a-4487-95b3-bac1cfeaa473 admin admin] PATH_INFO: `/v3/projects/df0d84f50c95401f88156fb6fef53f4d/users/f697ec059b7c4cb09c67ceca71a771e3/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:08.426201 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83c370-245a-4487-95b3-bac1cfeaa473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df0d84f50c95401f88156fb6fef53f4d, user_id=f697ec059b7c4cb09c67ceca71a771e3, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:08.429025 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83c370-245a-4487-95b3-bac1cfeaa473 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:08.466563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 991/2011] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:08 2022] PUT /identity/v3/projects/df0d84f50c95401f88156fb6fef53f4d/users/f697ec059b7c4cb09c67ceca71a771e3/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:08.475539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e6427-32e1-4b35-a1f0-7c192fb69201 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:08.475821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e6427-32e1-4b35-a1f0-7c192fb69201 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:08.476149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e6427-32e1-4b35-a1f0-7c192fb69201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:08.507817 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-ad7e6427-32e1-4b35-a1f0-7c192fb69201 None None] MFA Rules not processed for user `f697ec059b7c4cb09c67ceca71a771e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:08.509041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ad7e6427-32e1-4b35-a1f0-7c192fb69201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:08.559982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83c370-245a-4487-95b3-bac1cfeaa473 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:08.561720 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ed83c370-245a-4487-95b3-bac1cfeaa473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:08.625563 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1021/2012] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:08.631192 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e6427-32e1-4b35-a1f0-7c192fb69201 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:08.652989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83c370-245a-4487-95b3-bac1cfeaa473 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:08.658285 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f623d1-451c-4ce2-a95e-e26c7d9f5702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:08.659845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f623d1-451c-4ce2-a95e-e26c7d9f5702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:08.659845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f623d1-451c-4ce2-a95e-e26c7d9f5702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:08.660171 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f623d1-451c-4ce2-a95e-e26c7d9f5702 admin admin] PATH_INFO: `/v3/projects/20a5b0b324374f3da70074e32975d3d7` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:08.661278 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f623d1-451c-4ce2-a95e-e26c7d9f5702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20a5b0b324374f3da70074e32975d3d7)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:08.663192 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f623d1-451c-4ce2-a95e-e26c7d9f5702 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:08.693128 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:08.719386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-60f623d1-451c-4ce2-a95e-e26c7d9f5702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20a5b0b324374f3da70074e32975d3d7', 'request_id': 'req-60f623d1-451c-4ce2-a95e-e26c7d9f5702'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:08.722518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e6427-32e1-4b35-a1f0-7c192fb69201 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:08.731638 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-60f623d1-451c-4ce2-a95e-e26c7d9f5702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20a5b0b324374f3da70074e32975d3d7', 'request_id': 'req-60f623d1-451c-4ce2-a95e-e26c7d9f5702'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:08.741697 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-60f623d1-451c-4ce2-a95e-e26c7d9f5702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20a5b0b324374f3da70074e32975d3d7', 'request_id': 'req-60f623d1-451c-4ce2-a95e-e26c7d9f5702'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:08.754595 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-60f623d1-451c-4ce2-a95e-e26c7d9f5702 admin admin] The token cache is being invalidate because project 20a5b0b324374f3da70074e32975d3d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:08.754888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-60f623d1-451c-4ce2-a95e-e26c7d9f5702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:08.755568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-60f623d1-451c-4ce2-a95e-e26c7d9f5702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:08.757263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 992/2013] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:08 2022] DELETE /identity/v3/projects/20a5b0b324374f3da70074e32975d3d7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:08.761092 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f623d1-451c-4ce2-a95e-e26c7d9f5702 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:08.761977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:08.763239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-60f623d1-451c-4ce2-a95e-e26c7d9f5702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:08.787022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e41a6-67e8-48bf-a487-2436c6554406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:08.787876 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e41a6-67e8-48bf-a487-2436c6554406 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:08.788256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e41a6-67e8-48bf-a487-2436c6554406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:08.788555 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e41a6-67e8-48bf-a487-2436c6554406 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:08.789339 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e41a6-67e8-48bf-a487-2436c6554406 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:08.791555 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e41a6-67e8-48bf-a487-2436c6554406 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:08.822144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1022/2014] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:08 2022] POST /identity/v3/projects => generated 381 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:54:08.823236 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e41a6-67e8-48bf-a487-2436c6554406 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:08.824932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-099e41a6-67e8-48bf-a487-2436c6554406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:08.835206 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f623d1-451c-4ce2-a95e-e26c7d9f5702 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:08.897964 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe971d-70e4-44d9-b1df-f84240faba39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:08.898661 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe971d-70e4-44d9-b1df-f84240faba39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:08.898958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe971d-70e4-44d9-b1df-f84240faba39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:08.899281 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe971d-70e4-44d9-b1df-f84240faba39 admin admin] PATH_INFO: `/v3/users/ae72714f29b843caa9f7987362f62bd1` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:08.900485 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe971d-70e4-44d9-b1df-f84240faba39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae72714f29b843caa9f7987362f62bd1)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:08.902456 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe971d-70e4-44d9-b1df-f84240faba39 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:08.904294 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e41a6-67e8-48bf-a487-2436c6554406 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:08.910028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34909838-0dae-4bab-9512-1aba306bb568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:08.910863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34909838-0dae-4bab-9512-1aba306bb568 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:08.911214 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34909838-0dae-4bab-9512-1aba306bb568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:08.911560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34909838-0dae-4bab-9512-1aba306bb568 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:08.912392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34909838-0dae-4bab-9512-1aba306bb568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:08.914535 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34909838-0dae-4bab-9512-1aba306bb568 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:08.949959 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1023/2015] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:08 2022] POST /identity/v3/users => generated 297 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:08.950889 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34909838-0dae-4bab-9512-1aba306bb568 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:08.962143 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34909838-0dae-4bab-9512-1aba306bb568 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:08.967319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b75ab0-5907-4e22-960b-27abf4fccf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:08.967944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b75ab0-5907-4e22-960b-27abf4fccf7b admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:08.968227 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b75ab0-5907-4e22-960b-27abf4fccf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:08.968532 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b75ab0-5907-4e22-960b-27abf4fccf7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:08.969237 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b75ab0-5907-4e22-960b-27abf4fccf7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:08.970078 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-66fe971d-70e4-44d9-b1df-f84240faba39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae72714f29b843caa9f7987362f62bd1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:08.970864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b75ab0-5907-4e22-960b-27abf4fccf7b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:08.981365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:08.982341 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1024/2016] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:08.986265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b75ab0-5907-4e22-960b-27abf4fccf7b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:08.988376 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-66fe971d-70e4-44d9-b1df-f84240faba39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae72714f29b843caa9f7987362f62bd1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:08.997038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b75ab0-5907-4e22-960b-27abf4fccf7b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:09.002415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cd257-bf8c-4ad8-acc3-afb09dc2b502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:09.003198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cd257-bf8c-4ad8-acc3-afb09dc2b502 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:09.003660 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cd257-bf8c-4ad8-acc3-afb09dc2b502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:09.004052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cd257-bf8c-4ad8-acc3-afb09dc2b502 admin admin] PATH_INFO: `/v3/projects/d057f61777b64381be56d0d443d79e3f/users/dbf5e966b2b14367b3d1bfabe6e22f5c/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:09.008233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-66fe971d-70e4-44d9-b1df-f84240faba39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae72714f29b843caa9f7987362f62bd1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:09.009199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-66fe971d-70e4-44d9-b1df-f84240faba39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae72714f29b843caa9f7987362f62bd1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:09.021738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cd257-bf8c-4ad8-acc3-afb09dc2b502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d057f61777b64381be56d0d443d79e3f, user_id=dbf5e966b2b14367b3d1bfabe6e22f5c, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:09.024739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cd257-bf8c-4ad8-acc3-afb09dc2b502 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:09.034659 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-66fe971d-70e4-44d9-b1df-f84240faba39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae72714f29b843caa9f7987362f62bd1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:09.049287 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 993/2017] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:08 2022] DELETE /identity/v3/users/ae72714f29b843caa9f7987362f62bd1 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:09.058886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1025/2018] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:08 2022] PUT /identity/v3/projects/d057f61777b64381be56d0d443d79e3f/users/dbf5e966b2b14367b3d1bfabe6e22f5c/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:09.064132 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe971d-70e4-44d9-b1df-f84240faba39 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:09.071930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-66fe971d-70e4-44d9-b1df-f84240faba39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:09.146986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe971d-70e4-44d9-b1df-f84240faba39 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:09.206592 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea603115-1bbf-4f21-8bcb-9dd04b6e2087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:09.207263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea603115-1bbf-4f21-8bcb-9dd04b6e2087 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:09.207599 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea603115-1bbf-4f21-8bcb-9dd04b6e2087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:09.207902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea603115-1bbf-4f21-8bcb-9dd04b6e2087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:09.208651 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea603115-1bbf-4f21-8bcb-9dd04b6e2087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:09.210286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea603115-1bbf-4f21-8bcb-9dd04b6e2087 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:09.223754 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 994/2019] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:09 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:09.227700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cd257-bf8c-4ad8-acc3-afb09dc2b502 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:09.238978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cd257-bf8c-4ad8-acc3-afb09dc2b502 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:09.243661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2adad9-fae1-40d8-a650-5d14c6dfe155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:09.244326 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2adad9-fae1-40d8-a650-5d14c6dfe155 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:09.244679 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2adad9-fae1-40d8-a650-5d14c6dfe155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:09.244876 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2adad9-fae1-40d8-a650-5d14c6dfe155 admin admin] PATH_INFO: `/v3/projects/d057f61777b64381be56d0d443d79e3f/users/dbf5e966b2b14367b3d1bfabe6e22f5c/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:09.246943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2adad9-fae1-40d8-a650-5d14c6dfe155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d057f61777b64381be56d0d443d79e3f, user_id=dbf5e966b2b14367b3d1bfabe6e22f5c, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:09.249556 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2adad9-fae1-40d8-a650-5d14c6dfe155 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:09.259918 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1026/2020] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:09 2022] PUT /identity/v3/projects/d057f61777b64381be56d0d443d79e3f/users/dbf5e966b2b14367b3d1bfabe6e22f5c/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:09.270022 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc6895-2b1a-4fa3-a3cb-29ead0745947 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:09.270316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc6895-2b1a-4fa3-a3cb-29ead0745947 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:09.270628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc6895-2b1a-4fa3-a3cb-29ead0745947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:09.312672 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a3fc6895-2b1a-4fa3-a3cb-29ead0745947 None None] MFA Rules not processed for user `dbf5e966b2b14367b3d1bfabe6e22f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:09.313934 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a3fc6895-2b1a-4fa3-a3cb-29ead0745947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:09.384996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 995/2021] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:09.398343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2adad9-fae1-40d8-a650-5d14c6dfe155 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:09.400175 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ab2adad9-fae1-40d8-a650-5d14c6dfe155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:09.440607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc6895-2b1a-4fa3-a3cb-29ead0745947 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:09.442161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a3fc6895-2b1a-4fa3-a3cb-29ead0745947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:09.450332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:09.483425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2adad9-fae1-40d8-a650-5d14c6dfe155 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:09.487649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51023a36-2521-4287-834a-704e07f4f72f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:09.488548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51023a36-2521-4287-834a-704e07f4f72f service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:09.488548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51023a36-2521-4287-834a-704e07f4f72f service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:09.488789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51023a36-2521-4287-834a-704e07f4f72f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:09.499447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51023a36-2521-4287-834a-704e07f4f72f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:09.499959 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51023a36-2521-4287-834a-704e07f4f72f service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:09.509775 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1027/2022] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:09.535687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc6895-2b1a-4fa3-a3cb-29ead0745947 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:09.540855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a867e2-2d3c-4137-9ce8-0fb7183390b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:09.541487 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a867e2-2d3c-4137-9ce8-0fb7183390b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:09.541764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a867e2-2d3c-4137-9ce8-0fb7183390b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:09.542041 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a867e2-2d3c-4137-9ce8-0fb7183390b5 admin admin] PATH_INFO: `/v3/projects/944f2c5db12242bf95599e27efd7ce26` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:09.543289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a867e2-2d3c-4137-9ce8-0fb7183390b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=944f2c5db12242bf95599e27efd7ce26)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:09.545320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a867e2-2d3c-4137-9ce8-0fb7183390b5 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:09.626775 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d0a867e2-2d3c-4137-9ce8-0fb7183390b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '944f2c5db12242bf95599e27efd7ce26', 'request_id': 'req-d0a867e2-2d3c-4137-9ce8-0fb7183390b5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:09.650110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d0a867e2-2d3c-4137-9ce8-0fb7183390b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '944f2c5db12242bf95599e27efd7ce26', 'request_id': 'req-d0a867e2-2d3c-4137-9ce8-0fb7183390b5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:09.658136 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d0a867e2-2d3c-4137-9ce8-0fb7183390b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '944f2c5db12242bf95599e27efd7ce26', 'request_id': 'req-d0a867e2-2d3c-4137-9ce8-0fb7183390b5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:09.675027 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d0a867e2-2d3c-4137-9ce8-0fb7183390b5 admin admin] The token cache is being invalidate because project 944f2c5db12242bf95599e27efd7ce26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:09.675527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d0a867e2-2d3c-4137-9ce8-0fb7183390b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:09.676909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d0a867e2-2d3c-4137-9ce8-0fb7183390b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:09.679951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 996/2023] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:09 2022] DELETE /identity/v3/projects/944f2c5db12242bf95599e27efd7ce26 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:09.687121 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236fcd7-7df8-4ec5-a94a-8ed28dd392fb None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:09.687443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236fcd7-7df8-4ec5-a94a-8ed28dd392fb None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:09.687843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236fcd7-7df8-4ec5-a94a-8ed28dd392fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:09.704264 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:09.711843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-a236fcd7-7df8-4ec5-a94a-8ed28dd392fb None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:09.714347 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a236fcd7-7df8-4ec5-a94a-8ed28dd392fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:09.798551 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1028/2024] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:54:09.804329 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49990494-46d4-4832-9c12-322006bff08b None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:09.804740 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49990494-46d4-4832-9c12-322006bff08b None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:09.804996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49990494-46d4-4832-9c12-322006bff08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:09.827670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-49990494-46d4-4832-9c12-322006bff08b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:09.828857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-49990494-46d4-4832-9c12-322006bff08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:09.843201 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 997/2025] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:09.844360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236fcd7-7df8-4ec5-a94a-8ed28dd392fb None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:09.855316 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236fcd7-7df8-4ec5-a94a-8ed28dd392fb None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:09.878297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eadef6-c872-4438-9c7a-6d429784f55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:09.878297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eadef6-c872-4438-9c7a-6d429784f55d admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:09.878297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eadef6-c872-4438-9c7a-6d429784f55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:09.878297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eadef6-c872-4438-9c7a-6d429784f55d admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:09.878966 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eadef6-c872-4438-9c7a-6d429784f55d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:54:09.878966 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eadef6-c872-4438-9c7a-6d429784f55d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:09.880848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eadef6-c872-4438-9c7a-6d429784f55d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:09.893404 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1029/2026] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:54:09.898073 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49990494-46d4-4832-9c12-322006bff08b None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:09.904475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eadef6-c872-4438-9c7a-6d429784f55d admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:09.906273 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-21eadef6-c872-4438-9c7a-6d429784f55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:09.907784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49990494-46d4-4832-9c12-322006bff08b None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:09.912928 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0171e-dbcf-4f92-bcec-b40d50d2e2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:09.913542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0171e-dbcf-4f92-bcec-b40d50d2e2e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:09.913819 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0171e-dbcf-4f92-bcec-b40d50d2e2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:09.914094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0171e-dbcf-4f92-bcec-b40d50d2e2e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:09.914806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0171e-dbcf-4f92-bcec-b40d50d2e2e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:09.916672 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0171e-dbcf-4f92-bcec-b40d50d2e2e2 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:09.947739 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 998/2027] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:09 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:09.948716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0171e-dbcf-4f92-bcec-b40d50d2e2e2 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:09.973254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eadef6-c872-4438-9c7a-6d429784f55d admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:10.016196 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0171e-dbcf-4f92-bcec-b40d50d2e2e2 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:10.021448 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7a0b8-dd27-49b1-b048-07b932e700dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:10.022007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7a0b8-dd27-49b1-b048-07b932e700dc admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:10.022350 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7a0b8-dd27-49b1-b048-07b932e700dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:10.022699 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7a0b8-dd27-49b1-b048-07b932e700dc admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:10.023289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7a0b8-dd27-49b1-b048-07b932e700dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:10.025199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7a0b8-dd27-49b1-b048-07b932e700dc admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:10.035493 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca75eef-4350-4d35-b82d-c740e4f0eb54 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf5e966b2b14367b3d1bfabe6e22f5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd057f61777b64381be56d0d443d79e3f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:10.036123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca75eef-4350-4d35-b82d-c740e4f0eb54 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:10.036407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca75eef-4350-4d35-b82d-c740e4f0eb54 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:10.036694 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca75eef-4350-4d35-b82d-c740e4f0eb54 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:10.036966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:10.037389 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca75eef-4350-4d35-b82d-c740e4f0eb54 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:54:10.037698 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca75eef-4350-4d35-b82d-c740e4f0eb54 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:10.039547 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca75eef-4350-4d35-b82d-c740e4f0eb54 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:10.050160 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 999/2028] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:09 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:54:10.054109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7a0b8-dd27-49b1-b048-07b932e700dc admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:10.057415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1030/2029] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:54:10.061870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca75eef-4350-4d35-b82d-c740e4f0eb54 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:10.065900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7a0b8-dd27-49b1-b048-07b932e700dc admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:10.073675 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43692713-4b2b-4da9-86e3-31717c5bfdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:10.074195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca75eef-4350-4d35-b82d-c740e4f0eb54 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:10.074311 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43692713-4b2b-4da9-86e3-31717c5bfdaf admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:10.074597 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43692713-4b2b-4da9-86e3-31717c5bfdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:10.074880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43692713-4b2b-4da9-86e3-31717c5bfdaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:10.075562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43692713-4b2b-4da9-86e3-31717c5bfdaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:10.077511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43692713-4b2b-4da9-86e3-31717c5bfdaf admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:10.081207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6cdd9-dfd8-4345-8e90-516bc3faf4d7 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf5e966b2b14367b3d1bfabe6e22f5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd057f61777b64381be56d0d443d79e3f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:10.081861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6cdd9-dfd8-4345-8e90-516bc3faf4d7 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:10.082174 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6cdd9-dfd8-4345-8e90-516bc3faf4d7 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:10.083178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6cdd9-dfd8-4345-8e90-516bc3faf4d7 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:10.084433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6cdd9-dfd8-4345-8e90-516bc3faf4d7 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:10.087431 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e6cdd9-dfd8-4345-8e90-516bc3faf4d7 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:10.089391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1000/2030] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:54:10.095028 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43692713-4b2b-4da9-86e3-31717c5bfdaf admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:10.105482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43692713-4b2b-4da9-86e3-31717c5bfdaf admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:10.111026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e012a9-c776-4e58-96a7-4520bb2c2a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:10.111807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e012a9-c776-4e58-96a7-4520bb2c2a04 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:10.111983 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e012a9-c776-4e58-96a7-4520bb2c2a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:10.112205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e012a9-c776-4e58-96a7-4520bb2c2a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:10.112903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e012a9-c776-4e58-96a7-4520bb2c2a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:10.114622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e012a9-c776-4e58-96a7-4520bb2c2a04 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:10.121739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1031/2031] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:10 2022] POST /identity/v3/projects => generated 353 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:54:10.126206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6cdd9-dfd8-4345-8e90-516bc3faf4d7 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:10.128784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1001/2032] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:54:10.132911 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e012a9-c776-4e58-96a7-4520bb2c2a04 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:10.143003 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e012a9-c776-4e58-96a7-4520bb2c2a04 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:10.144022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:10.208412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6cdd9-dfd8-4345-8e90-516bc3faf4d7 tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:10.217650 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffefa7b-5049-4e39-90e9-a8aaff9d500e tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf5e966b2b14367b3d1bfabe6e22f5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd057f61777b64381be56d0d443d79e3f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:10.218288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffefa7b-5049-4e39-90e9-a8aaff9d500e tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:10.218613 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffefa7b-5049-4e39-90e9-a8aaff9d500e tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:10.218898 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffefa7b-5049-4e39-90e9-a8aaff9d500e tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:10.219616 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffefa7b-5049-4e39-90e9-a8aaff9d500e tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:54:10.219926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffefa7b-5049-4e39-90e9-a8aaff9d500e tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:10.221577 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffefa7b-5049-4e39-90e9-a8aaff9d500e tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:10.227004 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6268dc1-c4e0-4f0d-bc0f-dad48d8dd9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:10.227792 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6268dc1-c4e0-4f0d-bc0f-dad48d8dd9e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:10.227968 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6268dc1-c4e0-4f0d-bc0f-dad48d8dd9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:10.228257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6268dc1-c4e0-4f0d-bc0f-dad48d8dd9e1 admin admin] PATH_INFO: `/v3/projects/66dad5b3e9274645b5c52f3f3a78685b/users/7e1cf4e2fdc74d65ac5a99e6d6f5df52/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:10.256141 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1032/2033] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:54:10.270821 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6268dc1-c4e0-4f0d-bc0f-dad48d8dd9e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66dad5b3e9274645b5c52f3f3a78685b, user_id=7e1cf4e2fdc74d65ac5a99e6d6f5df52, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:10.273460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6268dc1-c4e0-4f0d-bc0f-dad48d8dd9e1 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:10.287824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1002/2034] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:10 2022] PUT /identity/v3/projects/66dad5b3e9274645b5c52f3f3a78685b/users/7e1cf4e2fdc74d65ac5a99e6d6f5df52/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:10.297772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffefa7b-5049-4e39-90e9-a8aaff9d500e tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:10.300713 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf896b-5b6c-4ca0-aecf-64404a6975bf None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:10.302126 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf896b-5b6c-4ca0-aecf-64404a6975bf None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:10.302650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf896b-5b6c-4ca0-aecf-64404a6975bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:10.331728 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffefa7b-5049-4e39-90e9-a8aaff9d500e tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:10.348001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-bdbf896b-5b6c-4ca0-aecf-64404a6975bf None None] MFA Rules not processed for user `7e1cf4e2fdc74d65ac5a99e6d6f5df52`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:10.349340 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-bdbf896b-5b6c-4ca0-aecf-64404a6975bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:10.427787 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41bb15-78f9-499c-bdc3-62e4a48330cf tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dbf5e966b2b14367b3d1bfabe6e22f5c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd057f61777b64381be56d0d443d79e3f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:10.428618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41bb15-78f9-499c-bdc3-62e4a48330cf tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:10.429432 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41bb15-78f9-499c-bdc3-62e4a48330cf tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:10.429731 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41bb15-78f9-499c-bdc3-62e4a48330cf tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] PATH_INFO: `/v3/projects/a1282178af634194ad2331c6027683d6` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:10.431026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41bb15-78f9-499c-bdc3-62e4a48330cf tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a1282178af634194ad2331c6027683d6)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:10.433178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd41bb15-78f9-499c-bdc3-62e4a48330cf tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:10.450630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1003/2035] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:10.453174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf896b-5b6c-4ca0-aecf-64404a6975bf None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:10.487043 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bd41bb15-78f9-499c-bdc3-62e4a48330cf tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1282178af634194ad2331c6027683d6', 'request_id': 'req-bd41bb15-78f9-499c-bdc3-62e4a48330cf'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:10.495764 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bd41bb15-78f9-499c-bdc3-62e4a48330cf tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1282178af634194ad2331c6027683d6', 'request_id': 'req-bd41bb15-78f9-499c-bdc3-62e4a48330cf'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:10.503023 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bd41bb15-78f9-499c-bdc3-62e4a48330cf tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1282178af634194ad2331c6027683d6', 'request_id': 'req-bd41bb15-78f9-499c-bdc3-62e4a48330cf'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:10.511380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:10.513958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bd41bb15-78f9-499c-bdc3-62e4a48330cf tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] The token cache is being invalidate because project a1282178af634194ad2331c6027683d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:10.514245 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bd41bb15-78f9-499c-bdc3-62e4a48330cf tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:10.514922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bd41bb15-78f9-499c-bdc3-62e4a48330cf tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:10.516512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1033/2036] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:10 2022] DELETE /identity/v3/projects/a1282178af634194ad2331c6027683d6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:10.527717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf896b-5b6c-4ca0-aecf-64404a6975bf None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:10.585031 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d0540-bb71-4fd4-9f46-1343c8635039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:10.585781 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d0540-bb71-4fd4-9f46-1343c8635039 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:10.586067 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d0540-bb71-4fd4-9f46-1343c8635039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:10.586363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d0540-bb71-4fd4-9f46-1343c8635039 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:10.587161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d0540-bb71-4fd4-9f46-1343c8635039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:10.589487 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d0540-bb71-4fd4-9f46-1343c8635039 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:10.626195 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1004/2037] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:10 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:10.631227 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41bb15-78f9-499c-bdc3-62e4a48330cf tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:10.632686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-bd41bb15-78f9-499c-bdc3-62e4a48330cf tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:10.649579 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41bb15-78f9-499c-bdc3-62e4a48330cf tempest-VolumeQuotaClassesTest-1099465163 tempest-VolumeQuotaClassesTest-1099465163-project-admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:10.694467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:10.726612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8f608-0f7a-43b1-b9aa-8af0a08c4944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:10.727253 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8f608-0f7a-43b1-b9aa-8af0a08c4944 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:10.727601 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8f608-0f7a-43b1-b9aa-8af0a08c4944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:10.727865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8f608-0f7a-43b1-b9aa-8af0a08c4944 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:10.728931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8f608-0f7a-43b1-b9aa-8af0a08c4944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:10.738047 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8f608-0f7a-43b1-b9aa-8af0a08c4944 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:10.760779 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d0540-bb71-4fd4-9f46-1343c8635039 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:10.762548 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-671d0540-bb71-4fd4-9f46-1343c8635039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:10.763969 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1034/2038] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:10 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:10.768703 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8f608-0f7a-43b1-b9aa-8af0a08c4944 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:10.776715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d0540-bb71-4fd4-9f46-1343c8635039 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:10.779739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8f608-0f7a-43b1-b9aa-8af0a08c4944 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:10.782800 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a264435-4691-4752-ab79-6369cb9bac4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:10.783556 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a264435-4691-4752-ab79-6369cb9bac4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:10.783902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a264435-4691-4752-ab79-6369cb9bac4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:10.784252 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a264435-4691-4752-ab79-6369cb9bac4f admin admin] PATH_INFO: `/v3/users/f697ec059b7c4cb09c67ceca71a771e3` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:10.785493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a264435-4691-4752-ab79-6369cb9bac4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f697ec059b7c4cb09c67ceca71a771e3)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:10.786451 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cafa3e-5828-46f0-a612-d66671ee5857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:10.787190 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cafa3e-5828-46f0-a612-d66671ee5857 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:10.787474 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cafa3e-5828-46f0-a612-d66671ee5857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:10.787740 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cafa3e-5828-46f0-a612-d66671ee5857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:10.787834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a264435-4691-4752-ab79-6369cb9bac4f admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:10.788441 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cafa3e-5828-46f0-a612-d66671ee5857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:10.790034 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cafa3e-5828-46f0-a612-d66671ee5857 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:10.804595 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1035/2039] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:54:10.805633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cafa3e-5828-46f0-a612-d66671ee5857 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:10.816232 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cafa3e-5828-46f0-a612-d66671ee5857 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:10.822593 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175f986-e580-4425-b561-03d71aef83bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:10.823286 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175f986-e580-4425-b561-03d71aef83bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:10.823610 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175f986-e580-4425-b561-03d71aef83bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:10.823890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175f986-e580-4425-b561-03d71aef83bd admin admin] PATH_INFO: `/v3/projects/459ba86cf53d405ca4323631082ee1f2/users/0d22056eef8240b7b285e8b8ec467cbe/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:10.841987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175f986-e580-4425-b561-03d71aef83bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=459ba86cf53d405ca4323631082ee1f2, user_id=0d22056eef8240b7b285e8b8ec467cbe, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:10.844596 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175f986-e580-4425-b561-03d71aef83bd admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:10.855389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-3a264435-4691-4752-ab79-6369cb9bac4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f697ec059b7c4cb09c67ceca71a771e3'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:10.865818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1036/2040] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:10 2022] PUT /identity/v3/projects/459ba86cf53d405ca4323631082ee1f2/users/0d22056eef8240b7b285e8b8ec467cbe/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:10.869845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175f986-e580-4425-b561-03d71aef83bd admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:10.870855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-3a264435-4691-4752-ab79-6369cb9bac4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f697ec059b7c4cb09c67ceca71a771e3'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:10.890268 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-3a264435-4691-4752-ab79-6369cb9bac4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f697ec059b7c4cb09c67ceca71a771e3'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:10.891227 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-3a264435-4691-4752-ab79-6369cb9bac4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f697ec059b7c4cb09c67ceca71a771e3'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:10.903250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-3a264435-4691-4752-ab79-6369cb9bac4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f697ec059b7c4cb09c67ceca71a771e3'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:10.910472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1005/2041] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:10 2022] DELETE /identity/v3/users/f697ec059b7c4cb09c67ceca71a771e3 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:10.916727 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ce395-c1c7-4729-b938-df92e00aed2c None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:10.917209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ce395-c1c7-4729-b938-df92e00aed2c None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:10.917492 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ce395-c1c7-4729-b938-df92e00aed2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:10.938509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-d89ce395-c1c7-4729-b938-df92e00aed2c None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:10.940275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d89ce395-c1c7-4729-b938-df92e00aed2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:10.945700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175f986-e580-4425-b561-03d71aef83bd admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:10.990245 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:11.011334 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1006/2042] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:54:11.012470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d90eaa-f186-4a03-b73a-426cd54185c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:11.013165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d90eaa-f186-4a03-b73a-426cd54185c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:11.013512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d90eaa-f186-4a03-b73a-426cd54185c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:11.013854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d90eaa-f186-4a03-b73a-426cd54185c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:11.014678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d90eaa-f186-4a03-b73a-426cd54185c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:11.016351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d90eaa-f186-4a03-b73a-426cd54185c6 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:11.017656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ce395-c1c7-4729-b938-df92e00aed2c None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:11.019707 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d89ce395-c1c7-4729-b938-df92e00aed2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:11.031199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1037/2043] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:54:11.032295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d90eaa-f186-4a03-b73a-426cd54185c6 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:11.034141 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-85d90eaa-f186-4a03-b73a-426cd54185c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:11.055069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d90eaa-f186-4a03-b73a-426cd54185c6 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:11.060086 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf464b-92fd-4ce4-a566-35e258c89e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:11.060891 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf464b-92fd-4ce4-a566-35e258c89e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:11.061270 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf464b-92fd-4ce4-a566-35e258c89e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:11.061749 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf464b-92fd-4ce4-a566-35e258c89e17 admin admin] PATH_INFO: `/v3/projects/459ba86cf53d405ca4323631082ee1f2/users/0d22056eef8240b7b285e8b8ec467cbe/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:11.066184 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf464b-92fd-4ce4-a566-35e258c89e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=459ba86cf53d405ca4323631082ee1f2, user_id=0d22056eef8240b7b285e8b8ec467cbe, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:11.068915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf464b-92fd-4ce4-a566-35e258c89e17 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:11.080022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1038/2044] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:11 2022] PUT /identity/v3/projects/459ba86cf53d405ca4323631082ee1f2/users/0d22056eef8240b7b285e8b8ec467cbe/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:11.090023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ce395-c1c7-4729-b938-df92e00aed2c None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:11.092142 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27955ca-ca61-4320-b5f7-0e544e211dda None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:11.092443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27955ca-ca61-4320-b5f7-0e544e211dda None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:11.092890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27955ca-ca61-4320-b5f7-0e544e211dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:11.136465 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388880d-8933-43e5-950c-168d852c153a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:11.137097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388880d-8933-43e5-950c-168d852c153a service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:11.137274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-f27955ca-ca61-4320-b5f7-0e544e211dda None None] MFA Rules not processed for user `0d22056eef8240b7b285e8b8ec467cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:11.137396 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388880d-8933-43e5-950c-168d852c153a service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:11.137638 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388880d-8933-43e5-950c-168d852c153a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:11.138463 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f27955ca-ca61-4320-b5f7-0e544e211dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:11.148511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:11.202064 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388880d-8933-43e5-950c-168d852c153a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:11.203636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388880d-8933-43e5-950c-168d852c153a service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:11.209558 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1039/2045] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:11.214845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1007/2046] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:54:11.220739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27955ca-ca61-4320-b5f7-0e544e211dda None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:11.222746 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f27955ca-ca61-4320-b5f7-0e544e211dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:11.296101 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27955ca-ca61-4320-b5f7-0e544e211dda None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:11.300769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0f537-5fc6-4798-a520-69f8d8b43dd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:11.301702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0f537-5fc6-4798-a520-69f8d8b43dd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:11.302029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0f537-5fc6-4798-a520-69f8d8b43dd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:11.302347 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0f537-5fc6-4798-a520-69f8d8b43dd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:11.305131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-11b0f537-5fc6-4798-a520-69f8d8b43dd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:11.370513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0f537-5fc6-4798-a520-69f8d8b43dd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:11.371872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b0f537-5fc6-4798-a520-69f8d8b43dd5 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:11.380798 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1040/2047] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:11.722970 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388880d-8933-43e5-950c-168d852c153a service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:11.728037 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f388880d-8933-43e5-950c-168d852c153a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:11.777463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388880d-8933-43e5-950c-168d852c153a service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:11.786022 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681f356-845b-4d42-b3eb-a7773b4a63f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:11.787320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681f356-845b-4d42-b3eb-a7773b4a63f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:11.787595 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681f356-845b-4d42-b3eb-a7773b4a63f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:11.787867 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9681f356-845b-4d42-b3eb-a7773b4a63f3 admin admin] PATH_INFO: `/v3/projects/df0d84f50c95401f88156fb6fef53f4d` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:11.789306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681f356-845b-4d42-b3eb-a7773b4a63f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df0d84f50c95401f88156fb6fef53f4d)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:11.791300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9681f356-845b-4d42-b3eb-a7773b4a63f3 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:11.884154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9681f356-845b-4d42-b3eb-a7773b4a63f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df0d84f50c95401f88156fb6fef53f4d', 'request_id': 'req-9681f356-845b-4d42-b3eb-a7773b4a63f3'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:11.888008 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:11.899594 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9681f356-845b-4d42-b3eb-a7773b4a63f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df0d84f50c95401f88156fb6fef53f4d', 'request_id': 'req-9681f356-845b-4d42-b3eb-a7773b4a63f3'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:11.916146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9681f356-845b-4d42-b3eb-a7773b4a63f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df0d84f50c95401f88156fb6fef53f4d', 'request_id': 'req-9681f356-845b-4d42-b3eb-a7773b4a63f3'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:11.929363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9681f356-845b-4d42-b3eb-a7773b4a63f3 admin admin] The token cache is being invalidate because project df0d84f50c95401f88156fb6fef53f4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:11.929658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9681f356-845b-4d42-b3eb-a7773b4a63f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:11.930440 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9681f356-845b-4d42-b3eb-a7773b4a63f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:11.932559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1008/2048] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:11 2022] DELETE /identity/v3/projects/df0d84f50c95401f88156fb6fef53f4d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:11.936475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0f537-5fc6-4798-a520-69f8d8b43dd5 service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:11.948802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0f537-5fc6-4798-a520-69f8d8b43dd5 service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:12.005532 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7273e23-436d-4b6d-98d9-1ee9c927b6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:12.006282 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7273e23-436d-4b6d-98d9-1ee9c927b6ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:12.006664 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7273e23-436d-4b6d-98d9-1ee9c927b6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:12.007038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7273e23-436d-4b6d-98d9-1ee9c927b6ca admin admin] PATH_INFO: `/v3/users/dbf5e966b2b14367b3d1bfabe6e22f5c` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:12.008306 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7273e23-436d-4b6d-98d9-1ee9c927b6ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbf5e966b2b14367b3d1bfabe6e22f5c)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:12.010234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7273e23-436d-4b6d-98d9-1ee9c927b6ca admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:12.013223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:12.067655 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f7273e23-436d-4b6d-98d9-1ee9c927b6ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbf5e966b2b14367b3d1bfabe6e22f5c'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:12.075984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f7273e23-436d-4b6d-98d9-1ee9c927b6ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbf5e966b2b14367b3d1bfabe6e22f5c'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:12.109778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f7273e23-436d-4b6d-98d9-1ee9c927b6ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbf5e966b2b14367b3d1bfabe6e22f5c'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:12.111780 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f7273e23-436d-4b6d-98d9-1ee9c927b6ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbf5e966b2b14367b3d1bfabe6e22f5c'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:12.112395 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f7273e23-436d-4b6d-98d9-1ee9c927b6ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbf5e966b2b14367b3d1bfabe6e22f5c'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:12.129034 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1041/2049] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:11 2022] DELETE /identity/v3/users/dbf5e966b2b14367b3d1bfabe6e22f5c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:12.515993 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681f356-845b-4d42-b3eb-a7773b4a63f3 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:12.517522 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-9681f356-845b-4d42-b3eb-a7773b4a63f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:12.531629 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9681f356-845b-4d42-b3eb-a7773b4a63f3 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:12.591462 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb833a-6674-4917-b338-9ed5cbde7491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:12.592117 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb833a-6674-4917-b338-9ed5cbde7491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:12.592400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb833a-6674-4917-b338-9ed5cbde7491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:12.592676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb833a-6674-4917-b338-9ed5cbde7491 admin admin] PATH_INFO: `/v3/projects/d057f61777b64381be56d0d443d79e3f` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:12.593984 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb833a-6674-4917-b338-9ed5cbde7491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d057f61777b64381be56d0d443d79e3f)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:12.596602 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb833a-6674-4917-b338-9ed5cbde7491 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:12.646628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-4feb833a-6674-4917-b338-9ed5cbde7491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd057f61777b64381be56d0d443d79e3f', 'request_id': 'req-4feb833a-6674-4917-b338-9ed5cbde7491'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:12.654577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-4feb833a-6674-4917-b338-9ed5cbde7491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd057f61777b64381be56d0d443d79e3f', 'request_id': 'req-4feb833a-6674-4917-b338-9ed5cbde7491'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:12.661633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-4feb833a-6674-4917-b338-9ed5cbde7491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd057f61777b64381be56d0d443d79e3f', 'request_id': 'req-4feb833a-6674-4917-b338-9ed5cbde7491'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:12.673810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-4feb833a-6674-4917-b338-9ed5cbde7491 admin admin] The token cache is being invalidate because project d057f61777b64381be56d0d443d79e3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:12.673944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-4feb833a-6674-4917-b338-9ed5cbde7491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:12.674566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-4feb833a-6674-4917-b338-9ed5cbde7491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:12.676446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1009/2050] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:12 2022] DELETE /identity/v3/projects/d057f61777b64381be56d0d443d79e3f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:12.681298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566a247-b0ac-4047-8fac-fbb4c903274e None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:12.681591 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566a247-b0ac-4047-8fac-fbb4c903274e None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:12.681872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566a247-b0ac-4047-8fac-fbb4c903274e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:12.702739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1566a247-b0ac-4047-8fac-fbb4c903274e None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:12.704095 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1566a247-b0ac-4047-8fac-fbb4c903274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:12.777853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1042/2051] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:12.786437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c4908-4316-40da-a2ea-397d0e75eb2b None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:12.786762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c4908-4316-40da-a2ea-397d0e75eb2b None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:12.787039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c4908-4316-40da-a2ea-397d0e75eb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:12.808991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-3a4c4908-4316-40da-a2ea-397d0e75eb2b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:12.810355 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-3a4c4908-4316-40da-a2ea-397d0e75eb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:12.834918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1010/2052] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:12.838900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566a247-b0ac-4047-8fac-fbb4c903274e None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:12.850856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566a247-b0ac-4047-8fac-fbb4c903274e None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:12.855806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e94a11-22c5-4953-a09a-f2a15425ab0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:12.862771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e94a11-22c5-4953-a09a-f2a15425ab0d admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:12.863107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e94a11-22c5-4953-a09a-f2a15425ab0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:12.863384 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e94a11-22c5-4953-a09a-f2a15425ab0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:12.864136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e94a11-22c5-4953-a09a-f2a15425ab0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:54:12.864445 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e94a11-22c5-4953-a09a-f2a15425ab0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:12.866076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e94a11-22c5-4953-a09a-f2a15425ab0d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:12.887093 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1043/2053] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:54:12.891401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c4908-4316-40da-a2ea-397d0e75eb2b None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:12.903339 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c4908-4316-40da-a2ea-397d0e75eb2b None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:12.908925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0bc49-3a5b-4984-990d-bd032fe3ae35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:12.909558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0bc49-3a5b-4984-990d-bd032fe3ae35 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:12.909842 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0bc49-3a5b-4984-990d-bd032fe3ae35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:12.910118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee0bc49-3a5b-4984-990d-bd032fe3ae35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:12.910846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0bc49-3a5b-4984-990d-bd032fe3ae35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:12.913144 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee0bc49-3a5b-4984-990d-bd032fe3ae35 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:12.950244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1011/2054] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:12 2022] POST /identity/v3/projects => generated 367 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:12.954156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e94a11-22c5-4953-a09a-f2a15425ab0d admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:12.973502 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e94a11-22c5-4953-a09a-f2a15425ab0d admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:13.083819 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aef71b-f119-4dd2-8cac-bd668a52c6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:13.085230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aef71b-f119-4dd2-8cac-bd668a52c6ac admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:13.085526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aef71b-f119-4dd2-8cac-bd668a52c6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:13.085803 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aef71b-f119-4dd2-8cac-bd668a52c6ac admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:13.088165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aef71b-f119-4dd2-8cac-bd668a52c6ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:13.090100 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aef71b-f119-4dd2-8cac-bd668a52c6ac admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:13.109207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:13.118661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1044/2055] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:12 2022] POST /identity/v3/users => generated 284 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:54:13.131434 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0bc49-3a5b-4984-990d-bd032fe3ae35 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:13.144590 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee0bc49-3a5b-4984-990d-bd032fe3ae35 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:13.151168 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f6059-fec7-4cdd-bc9b-cb5859eaea71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:13.152363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f6059-fec7-4cdd-bc9b-cb5859eaea71 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:13.152876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f6059-fec7-4cdd-bc9b-cb5859eaea71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:13.153534 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f6059-fec7-4cdd-bc9b-cb5859eaea71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:13.154505 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f6059-fec7-4cdd-bc9b-cb5859eaea71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:13.156979 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f6059-fec7-4cdd-bc9b-cb5859eaea71 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:13.172620 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1012/2056] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:13.177406 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aef71b-f119-4dd2-8cac-bd668a52c6ac admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:13.188734 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aef71b-f119-4dd2-8cac-bd668a52c6ac admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:13.194064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8b8d8-29c6-4e1d-a15e-746f6b1ee01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:13.194783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8b8d8-29c6-4e1d-a15e-746f6b1ee01e admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:13.196438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8b8d8-29c6-4e1d-a15e-746f6b1ee01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:13.196438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8b8d8-29c6-4e1d-a15e-746f6b1ee01e admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:13.196926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8b8d8-29c6-4e1d-a15e-746f6b1ee01e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:13.199409 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8b8d8-29c6-4e1d-a15e-746f6b1ee01e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:13.236221 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1045/2057] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:13.240932 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f6059-fec7-4cdd-bc9b-cb5859eaea71 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:13.255116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f6059-fec7-4cdd-bc9b-cb5859eaea71 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:13.269559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcbc85-651c-4d6e-b787-d26f61748b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:13.269559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcbc85-651c-4d6e-b787-d26f61748b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:13.269559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcbc85-651c-4d6e-b787-d26f61748b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:13.269938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fcbc85-651c-4d6e-b787-d26f61748b50 admin admin] PATH_INFO: `/v3/projects/8dab965a8b2544638322510c199897b6/users/478be52934d34e6b8baf21b0375ed4d6/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:13.307715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fcbc85-651c-4d6e-b787-d26f61748b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dab965a8b2544638322510c199897b6, user_id=478be52934d34e6b8baf21b0375ed4d6, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:13.310800 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fcbc85-651c-4d6e-b787-d26f61748b50 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:13.345109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1013/2058] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:13 2022] PUT /identity/v3/projects/8dab965a8b2544638322510c199897b6/users/478be52934d34e6b8baf21b0375ed4d6/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:13.356946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cec7c8-8361-4938-917f-3aeacd5caa4c None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:13.357254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cec7c8-8361-4938-917f-3aeacd5caa4c None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:13.357505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cec7c8-8361-4938-917f-3aeacd5caa4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:13.398824 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-43cec7c8-8361-4938-917f-3aeacd5caa4c None None] MFA Rules not processed for user `478be52934d34e6b8baf21b0375ed4d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:13.400117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-43cec7c8-8361-4938-917f-3aeacd5caa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:13.497562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1046/2059] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:13.502731 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcbc85-651c-4d6e-b787-d26f61748b50 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:13.517201 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:13.617926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fcbc85-651c-4d6e-b787-d26f61748b50 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:13.622935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67313b5f-a35f-475b-9cec-41d10cb38861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:13.623568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67313b5f-a35f-475b-9cec-41d10cb38861 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:13.623854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67313b5f-a35f-475b-9cec-41d10cb38861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:13.624128 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67313b5f-a35f-475b-9cec-41d10cb38861 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:13.624804 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67313b5f-a35f-475b-9cec-41d10cb38861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:13.626907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67313b5f-a35f-475b-9cec-41d10cb38861 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:13.659120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1014/2060] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:13 2022] POST /identity/v3/projects => generated 367 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:13.663183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cec7c8-8361-4938-917f-3aeacd5caa4c None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:13.678413 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cec7c8-8361-4938-917f-3aeacd5caa4c None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:13.758267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714d313-95c7-4c3d-bd39-184ee112c604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:13.758852 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714d313-95c7-4c3d-bd39-184ee112c604 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:13.759663 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714d313-95c7-4c3d-bd39-184ee112c604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:13.759959 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714d313-95c7-4c3d-bd39-184ee112c604 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:13.760813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714d313-95c7-4c3d-bd39-184ee112c604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:13.763276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714d313-95c7-4c3d-bd39-184ee112c604 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:13.795018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1047/2061] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:13 2022] POST /identity/v3/users => generated 290 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:13.799784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67313b5f-a35f-475b-9cec-41d10cb38861 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:13.814001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67313b5f-a35f-475b-9cec-41d10cb38861 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:13.820528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25df9e-bcb2-4ac6-ab9b-ebf92e9ae48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:13.821499 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25df9e-bcb2-4ac6-ab9b-ebf92e9ae48b admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:13.822266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25df9e-bcb2-4ac6-ab9b-ebf92e9ae48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:13.822865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25df9e-bcb2-4ac6-ab9b-ebf92e9ae48b admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:13.823849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25df9e-bcb2-4ac6-ab9b-ebf92e9ae48b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:13.825751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25df9e-bcb2-4ac6-ab9b-ebf92e9ae48b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:13.839169 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1015/2062] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:13.844293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714d313-95c7-4c3d-bd39-184ee112c604 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:13.862593 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714d313-95c7-4c3d-bd39-184ee112c604 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:13.869914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28bff1-c800-4981-a4c9-44ed07fff2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:13.871036 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28bff1-c800-4981-a4c9-44ed07fff2cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:13.871324 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28bff1-c800-4981-a4c9-44ed07fff2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:13.871645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28bff1-c800-4981-a4c9-44ed07fff2cb admin admin] PATH_INFO: `/v3/projects/c1761ae5e8b541499e595b0fc9bcd939/users/7b5b5254fbb7451a9c5c66f24446aacc/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:13.891846 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28bff1-c800-4981-a4c9-44ed07fff2cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1761ae5e8b541499e595b0fc9bcd939, user_id=7b5b5254fbb7451a9c5c66f24446aacc, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:13.894932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28bff1-c800-4981-a4c9-44ed07fff2cb admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:13.922611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1048/2063] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:13 2022] PUT /identity/v3/projects/c1761ae5e8b541499e595b0fc9bcd939/users/7b5b5254fbb7451a9c5c66f24446aacc/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:13.925333 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25df9e-bcb2-4ac6-ab9b-ebf92e9ae48b admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:13.940710 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25df9e-bcb2-4ac6-ab9b-ebf92e9ae48b admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:14.044284 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8832b16-8a08-4b67-828b-3e5e43246a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:14.045563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8832b16-8a08-4b67-828b-3e5e43246a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:14.046055 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8832b16-8a08-4b67-828b-3e5e43246a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:14.046601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8832b16-8a08-4b67-828b-3e5e43246a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:14.047858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8832b16-8a08-4b67-828b-3e5e43246a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:14.050274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8832b16-8a08-4b67-828b-3e5e43246a0e admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:14.067765 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1016/2064] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:13 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:14.071416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28bff1-c800-4981-a4c9-44ed07fff2cb admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:14.083369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28bff1-c800-4981-a4c9-44ed07fff2cb admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:14.105796 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdeb1c-3aa0-4a99-95d1-8cb11880a745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:14.107970 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdeb1c-3aa0-4a99-95d1-8cb11880a745 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:14.108475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdeb1c-3aa0-4a99-95d1-8cb11880a745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:14.109731 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdeb1c-3aa0-4a99-95d1-8cb11880a745 admin admin] PATH_INFO: `/v3/projects/c1761ae5e8b541499e595b0fc9bcd939/users/7b5b5254fbb7451a9c5c66f24446aacc/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:14.115323 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdeb1c-3aa0-4a99-95d1-8cb11880a745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1761ae5e8b541499e595b0fc9bcd939, user_id=7b5b5254fbb7451a9c5c66f24446aacc, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:14.121500 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdeb1c-3aa0-4a99-95d1-8cb11880a745 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:14.150029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1049/2065] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:14 2022] PUT /identity/v3/projects/c1761ae5e8b541499e595b0fc9bcd939/users/7b5b5254fbb7451a9c5c66f24446aacc/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:14.160499 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b3eee-ebd7-4a0c-b632-c1cd5778f8bc None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:14.161387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b3eee-ebd7-4a0c-b632-c1cd5778f8bc None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:14.161750 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b3eee-ebd7-4a0c-b632-c1cd5778f8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:14.208907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-ca2b3eee-ebd7-4a0c-b632-c1cd5778f8bc None None] MFA Rules not processed for user `7b5b5254fbb7451a9c5c66f24446aacc`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:14.210229 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ca2b3eee-ebd7-4a0c-b632-c1cd5778f8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:14.323319 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1017/2066] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:14.332114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdeb1c-3aa0-4a99-95d1-8cb11880a745 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:14.334182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-3dbdeb1c-3aa0-4a99-95d1-8cb11880a745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:14.352338 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b3eee-ebd7-4a0c-b632-c1cd5778f8bc None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:14.354123 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ca2b3eee-ebd7-4a0c-b632-c1cd5778f8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:14.360659 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:14.421220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:14.474222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdeb1c-3aa0-4a99-95d1-8cb11880a745 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:14.476806 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b3eee-ebd7-4a0c-b632-c1cd5778f8bc None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:14.483157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef64bc1-1a4a-4640-a163-c0f5b1bee1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:14.483970 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef64bc1-1a4a-4640-a163-c0f5b1bee1c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:14.484325 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef64bc1-1a4a-4640-a163-c0f5b1bee1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:14.484669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef64bc1-1a4a-4640-a163-c0f5b1bee1c6 admin admin] PATH_INFO: `/v3/users/7e1cf4e2fdc74d65ac5a99e6d6f5df52` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:14.485956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef64bc1-1a4a-4640-a163-c0f5b1bee1c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e1cf4e2fdc74d65ac5a99e6d6f5df52)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:14.486952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4959732-21cf-4592-83b8-04991c3a3593 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:14.487594 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4959732-21cf-4592-83b8-04991c3a3593 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:14.487876 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4959732-21cf-4592-83b8-04991c3a3593 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:14.488149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4959732-21cf-4592-83b8-04991c3a3593 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:14.488334 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef64bc1-1a4a-4640-a163-c0f5b1bee1c6 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:14.511461 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4959732-21cf-4592-83b8-04991c3a3593 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:14.513792 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4959732-21cf-4592-83b8-04991c3a3593 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:14.532697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1050/2067] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:14.594368 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-3ef64bc1-1a4a-4640-a163-c0f5b1bee1c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e1cf4e2fdc74d65ac5a99e6d6f5df52'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:14.635448 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-3ef64bc1-1a4a-4640-a163-c0f5b1bee1c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e1cf4e2fdc74d65ac5a99e6d6f5df52'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:14.648508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-3ef64bc1-1a4a-4640-a163-c0f5b1bee1c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e1cf4e2fdc74d65ac5a99e6d6f5df52'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:14.649339 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-3ef64bc1-1a4a-4640-a163-c0f5b1bee1c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e1cf4e2fdc74d65ac5a99e6d6f5df52'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:14.659499 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-3ef64bc1-1a4a-4640-a163-c0f5b1bee1c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e1cf4e2fdc74d65ac5a99e6d6f5df52'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:14.671646 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1018/2068] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:14 2022] DELETE /identity/v3/users/7e1cf4e2fdc74d65ac5a99e6d6f5df52 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:14.677567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a166e-8916-4c45-a398-0a1cf99c226b None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:14.678664 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a166e-8916-4c45-a398-0a1cf99c226b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:14.679165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a166e-8916-4c45-a398-0a1cf99c226b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:14.679443 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef64bc1-1a4a-4640-a163-c0f5b1bee1c6 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:14.684576 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-3ef64bc1-1a4a-4640-a163-c0f5b1bee1c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:14.750758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-bf0a166e-8916-4c45-a398-0a1cf99c226b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:14.753256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-bf0a166e-8916-4c45-a398-0a1cf99c226b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:14.802260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef64bc1-1a4a-4640-a163-c0f5b1bee1c6 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:14.808059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdc472-c119-4cd7-b02e-275973f80d84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:14.808807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdc472-c119-4cd7-b02e-275973f80d84 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:14.809100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdc472-c119-4cd7-b02e-275973f80d84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:14.809497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdc472-c119-4cd7-b02e-275973f80d84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:14.811924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2bbdc472-c119-4cd7-b02e-275973f80d84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:14.873962 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:14.895951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1051/2069] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:14.949090 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbdc472-c119-4cd7-b02e-275973f80d84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:14.951071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbdc472-c119-4cd7-b02e-275973f80d84 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:14.953757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a166e-8916-4c45-a398-0a1cf99c226b None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:14.956532 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-bf0a166e-8916-4c45-a398-0a1cf99c226b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:14.969846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1019/2070] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:54:15.032097 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a166e-8916-4c45-a398-0a1cf99c226b None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:15.043862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59c98f-2859-4eee-8b6d-ef0e9408cbdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:15.044537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59c98f-2859-4eee-8b6d-ef0e9408cbdb service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:15.044778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59c98f-2859-4eee-8b6d-ef0e9408cbdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:15.045054 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59c98f-2859-4eee-8b6d-ef0e9408cbdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:15.068364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d59c98f-2859-4eee-8b6d-ef0e9408cbdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:15.070359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d59c98f-2859-4eee-8b6d-ef0e9408cbdb service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:15.101207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1052/2071] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:54:15.566392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdc472-c119-4cd7-b02e-275973f80d84 service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:15.568020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2bbdc472-c119-4cd7-b02e-275973f80d84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:15.599844 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdc472-c119-4cd7-b02e-275973f80d84 service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:15.604820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:15.605478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:15.605764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:15.606037 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8 admin admin] PATH_INFO: `/v3/projects/66dad5b3e9274645b5c52f3f3a78685b` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:15.611320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66dad5b3e9274645b5c52f3f3a78685b)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:15.613298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:15.696767 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66dad5b3e9274645b5c52f3f3a78685b', 'request_id': 'req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:15.705016 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66dad5b3e9274645b5c52f3f3a78685b', 'request_id': 'req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:15.712231 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66dad5b3e9274645b5c52f3f3a78685b', 'request_id': 'req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:15.724100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8 admin admin] The token cache is being invalidate because project 66dad5b3e9274645b5c52f3f3a78685b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:15.724503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:15.725239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:15.727134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1020/2072] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:15 2022] DELETE /identity/v3/projects/66dad5b3e9274645b5c52f3f3a78685b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:15.731904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59c98f-2859-4eee-8b6d-ef0e9408cbdb service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:15.741250 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59c98f-2859-4eee-8b6d-ef0e9408cbdb service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:15.812536 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a83888-5c01-4fcf-ab6d-7d900d83e029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:15.813261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a83888-5c01-4fcf-ab6d-7d900d83e029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:15.813626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a83888-5c01-4fcf-ab6d-7d900d83e029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:15.813974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a83888-5c01-4fcf-ab6d-7d900d83e029 admin admin] PATH_INFO: `/v3/users/0d22056eef8240b7b285e8b8ec467cbe` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:15.815292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a83888-5c01-4fcf-ab6d-7d900d83e029 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d22056eef8240b7b285e8b8ec467cbe)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:15.817309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a83888-5c01-4fcf-ab6d-7d900d83e029 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:15.837316 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:15.885586 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-00a83888-5c01-4fcf-ab6d-7d900d83e029 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d22056eef8240b7b285e8b8ec467cbe'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:15.895786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-00a83888-5c01-4fcf-ab6d-7d900d83e029 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d22056eef8240b7b285e8b8ec467cbe'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:15.910771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-00a83888-5c01-4fcf-ab6d-7d900d83e029 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d22056eef8240b7b285e8b8ec467cbe'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:15.911520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-00a83888-5c01-4fcf-ab6d-7d900d83e029 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d22056eef8240b7b285e8b8ec467cbe'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:15.912135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-00a83888-5c01-4fcf-ab6d-7d900d83e029 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d22056eef8240b7b285e8b8ec467cbe'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:15.929504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1053/2073] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:15 2022] DELETE /identity/v3/users/0d22056eef8240b7b285e8b8ec467cbe => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:16.324526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:16.326256 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:16.364138 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dfdff3-e35b-421a-a761-3bdfc1d8c5f8 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:16.435462 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8169a245-abc5-4d38-b710-4739dd53b30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:16.436230 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169a245-abc5-4d38-b710-4739dd53b30e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:16.436514 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169a245-abc5-4d38-b710-4739dd53b30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:16.436790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8169a245-abc5-4d38-b710-4739dd53b30e admin admin] PATH_INFO: `/v3/projects/459ba86cf53d405ca4323631082ee1f2` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:16.438036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8169a245-abc5-4d38-b710-4739dd53b30e admin admin] RBAC: Authorizing `identity:delete_project(project_id=459ba86cf53d405ca4323631082ee1f2)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:16.440599 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8169a245-abc5-4d38-b710-4739dd53b30e admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:16.497570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8169a245-abc5-4d38-b710-4739dd53b30e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '459ba86cf53d405ca4323631082ee1f2', 'request_id': 'req-8169a245-abc5-4d38-b710-4739dd53b30e'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:16.511382 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8169a245-abc5-4d38-b710-4739dd53b30e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '459ba86cf53d405ca4323631082ee1f2', 'request_id': 'req-8169a245-abc5-4d38-b710-4739dd53b30e'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:16.521312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8169a245-abc5-4d38-b710-4739dd53b30e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '459ba86cf53d405ca4323631082ee1f2', 'request_id': 'req-8169a245-abc5-4d38-b710-4739dd53b30e'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:16.523023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:16.536827 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8169a245-abc5-4d38-b710-4739dd53b30e admin admin] The token cache is being invalidate because project 459ba86cf53d405ca4323631082ee1f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:16.537114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8169a245-abc5-4d38-b710-4739dd53b30e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:16.538027 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8169a245-abc5-4d38-b710-4739dd53b30e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:16.539894 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1021/2074] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:16 2022] DELETE /identity/v3/projects/459ba86cf53d405ca4323631082ee1f2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:16.544705 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc23f6-7bdd-4449-9b22-a51c2abac537 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:16.545188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc23f6-7bdd-4449-9b22-a51c2abac537 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:16.545282 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc23f6-7bdd-4449-9b22-a51c2abac537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:16.565580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-5afc23f6-7bdd-4449-9b22-a51c2abac537 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:16.566819 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5afc23f6-7bdd-4449-9b22-a51c2abac537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:16.634063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1054/2075] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:16.638286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093f060-58a0-41d5-bd44-182e0f36c7cc None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:16.638827 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093f060-58a0-41d5-bd44-182e0f36c7cc None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:16.638926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1093f060-58a0-41d5-bd44-182e0f36c7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:16.666234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-1093f060-58a0-41d5-bd44-182e0f36c7cc None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:16.667671 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1093f060-58a0-41d5-bd44-182e0f36c7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:16.678168 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1022/2076] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:16.682211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc23f6-7bdd-4449-9b22-a51c2abac537 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:16.697300 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc23f6-7bdd-4449-9b22-a51c2abac537 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:16.713087 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fce26-dc5b-4c13-bbd1-42799af2cb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:16.714182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fce26-dc5b-4c13-bbd1-42799af2cb0d admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:16.714468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fce26-dc5b-4c13-bbd1-42799af2cb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:16.719240 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39fce26-dc5b-4c13-bbd1-42799af2cb0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:16.719240 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fce26-dc5b-4c13-bbd1-42799af2cb0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:54:16.719240 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fce26-dc5b-4c13-bbd1-42799af2cb0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:16.719240 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39fce26-dc5b-4c13-bbd1-42799af2cb0d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:16.735465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1055/2077] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:54:16.740201 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093f060-58a0-41d5-bd44-182e0f36c7cc None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:16.750397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1093f060-58a0-41d5-bd44-182e0f36c7cc None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:16.756176 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a8f42-d427-41dd-8b2e-203b475a4359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:16.756727 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a8f42-d427-41dd-8b2e-203b475a4359 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:16.757154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a8f42-d427-41dd-8b2e-203b475a4359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:16.757281 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a8f42-d427-41dd-8b2e-203b475a4359 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:16.757959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a8f42-d427-41dd-8b2e-203b475a4359 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:16.759886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a8f42-d427-41dd-8b2e-203b475a4359 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:16.801797 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1023/2078] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:16 2022] POST /identity/v3/projects => generated 399 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:16.803063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fce26-dc5b-4c13-bbd1-42799af2cb0d admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:16.813521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39fce26-dc5b-4c13-bbd1-42799af2cb0d admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:16.891333 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311fd06d-5546-49e3-ad5d-e4ac76378a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:16.891981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fd06d-5546-49e3-ad5d-e4ac76378a8e admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:16.892259 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fd06d-5546-49e3-ad5d-e4ac76378a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:16.892531 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311fd06d-5546-49e3-ad5d-e4ac76378a8e admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:16.894355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311fd06d-5546-49e3-ad5d-e4ac76378a8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:16.895870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311fd06d-5546-49e3-ad5d-e4ac76378a8e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:16.930374 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1056/2079] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:16 2022] POST /identity/v3/users => generated 300 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:16.934311 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a8f42-d427-41dd-8b2e-203b475a4359 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:16.946978 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a8f42-d427-41dd-8b2e-203b475a4359 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:16.956133 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140c322-2cb1-472e-a9a3-1405105af438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:16.956728 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140c322-2cb1-472e-a9a3-1405105af438 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:16.957011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140c322-2cb1-472e-a9a3-1405105af438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:16.957284 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140c322-2cb1-472e-a9a3-1405105af438 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:16.958309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140c322-2cb1-472e-a9a3-1405105af438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:16.963401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140c322-2cb1-472e-a9a3-1405105af438 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:16.983254 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1024/2080] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:16.987056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311fd06d-5546-49e3-ad5d-e4ac76378a8e admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:17.000074 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311fd06d-5546-49e3-ad5d-e4ac76378a8e admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:17.006068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d46b5e4-96ab-42aa-aded-359f0923670d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:17.006792 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46b5e4-96ab-42aa-aded-359f0923670d admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:17.007077 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46b5e4-96ab-42aa-aded-359f0923670d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:17.007404 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d46b5e4-96ab-42aa-aded-359f0923670d admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:17.008143 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46b5e4-96ab-42aa-aded-359f0923670d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:17.009806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d46b5e4-96ab-42aa-aded-359f0923670d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:17.020455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:17.024727 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1057/2081] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:17.028567 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140c322-2cb1-472e-a9a3-1405105af438 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:17.040659 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140c322-2cb1-472e-a9a3-1405105af438 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:17.046112 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31756e5d-f904-48b9-97ce-1961800c5a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:17.046837 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31756e5d-f904-48b9-97ce-1961800c5a45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:17.047118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31756e5d-f904-48b9-97ce-1961800c5a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:17.047393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31756e5d-f904-48b9-97ce-1961800c5a45 admin admin] PATH_INFO: `/v3/projects/0de87c3f70a84e23838af32e7c546e88/users/272840ea732c4291ba561e033796f301/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:17.064377 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31756e5d-f904-48b9-97ce-1961800c5a45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0de87c3f70a84e23838af32e7c546e88, user_id=272840ea732c4291ba561e033796f301, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:17.067681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31756e5d-f904-48b9-97ce-1961800c5a45 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:17.082103 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1025/2082] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:17 2022] PUT /identity/v3/projects/0de87c3f70a84e23838af32e7c546e88/users/272840ea732c4291ba561e033796f301/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:17.093048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951de854-5daf-4bd7-af08-660d2903338d None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:17.093341 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951de854-5daf-4bd7-af08-660d2903338d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:17.093612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951de854-5daf-4bd7-af08-660d2903338d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:17.124576 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-951de854-5daf-4bd7-af08-660d2903338d None None] MFA Rules not processed for user `272840ea732c4291ba561e033796f301`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:17.125727 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-951de854-5daf-4bd7-af08-660d2903338d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:17.215787 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1058/2083] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:17 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:54:17.230850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31756e5d-f904-48b9-97ce-1961800c5a45 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:17.232485 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-31756e5d-f904-48b9-97ce-1961800c5a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:17.297316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31756e5d-f904-48b9-97ce-1961800c5a45 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:17.301540 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf66e83-8610-4b91-bd7d-e7feee427f2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:17.302171 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf66e83-8610-4b91-bd7d-e7feee427f2d service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:17.302452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf66e83-8610-4b91-bd7d-e7feee427f2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:17.302767 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf66e83-8610-4b91-bd7d-e7feee427f2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:17.312472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf66e83-8610-4b91-bd7d-e7feee427f2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:17.313938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf66e83-8610-4b91-bd7d-e7feee427f2d service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:17.325810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1026/2084] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:17 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:24.501357 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951de854-5daf-4bd7-af08-660d2903338d None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:24.503274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-951de854-5daf-4bd7-af08-660d2903338d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:24.568940 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951de854-5daf-4bd7-af08-660d2903338d None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:24.573435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d5c88-0f55-4a79-a53b-006e3b0b0e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:24.574063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d5c88-0f55-4a79-a53b-006e3b0b0e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:24.574340 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d5c88-0f55-4a79-a53b-006e3b0b0e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:24.574643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d5c88-0f55-4a79-a53b-006e3b0b0e53 admin admin] PATH_INFO: `/v3/users/478be52934d34e6b8baf21b0375ed4d6` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:24.575778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d5c88-0f55-4a79-a53b-006e3b0b0e53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=478be52934d34e6b8baf21b0375ed4d6)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:24.577668 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d5c88-0f55-4a79-a53b-006e3b0b0e53 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:24.621309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-886d5c88-0f55-4a79-a53b-006e3b0b0e53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '478be52934d34e6b8baf21b0375ed4d6'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:24.629778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-886d5c88-0f55-4a79-a53b-006e3b0b0e53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '478be52934d34e6b8baf21b0375ed4d6'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:24.641421 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-886d5c88-0f55-4a79-a53b-006e3b0b0e53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '478be52934d34e6b8baf21b0375ed4d6'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:24.642121 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-886d5c88-0f55-4a79-a53b-006e3b0b0e53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '478be52934d34e6b8baf21b0375ed4d6'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:24.642850 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-886d5c88-0f55-4a79-a53b-006e3b0b0e53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '478be52934d34e6b8baf21b0375ed4d6'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:24.658874 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1059/2085] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:24 2022] DELETE /identity/v3/users/478be52934d34e6b8baf21b0375ed4d6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:24.663857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76dfe7-dc55-464f-ab55-b149c98b16d0 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:24.664431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76dfe7-dc55-464f-ab55-b149c98b16d0 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:24.664679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76dfe7-dc55-464f-ab55-b149c98b16d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:24.686665 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-8e76dfe7-dc55-464f-ab55-b149c98b16d0 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:24.687902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8e76dfe7-dc55-464f-ab55-b149c98b16d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:24.757293 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1027/2086] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:54:24.767111 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d5c88-0f55-4a79-a53b-006e3b0b0e53 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:24.770686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-886d5c88-0f55-4a79-a53b-006e3b0b0e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:24.783081 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:24.841967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d5c88-0f55-4a79-a53b-006e3b0b0e53 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:24.845643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49618886-0b14-43dd-8e7c-31f2790c86ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:24.846280 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49618886-0b14-43dd-8e7c-31f2790c86ab service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:24.846599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49618886-0b14-43dd-8e7c-31f2790c86ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:24.846918 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49618886-0b14-43dd-8e7c-31f2790c86ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:24.856220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49618886-0b14-43dd-8e7c-31f2790c86ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:24.857679 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49618886-0b14-43dd-8e7c-31f2790c86ab service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:24.868196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1060/2087] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:25.134198 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76dfe7-dc55-464f-ab55-b149c98b16d0 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:25.135790 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8e76dfe7-dc55-464f-ab55-b149c98b16d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:25.150832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76dfe7-dc55-464f-ab55-b149c98b16d0 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:25.155886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:25.156478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:25.156753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:25.157047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7 admin admin] PATH_INFO: `/v3/projects/8dab965a8b2544638322510c199897b6` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:25.158202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dab965a8b2544638322510c199897b6)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:25.160701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:25.185213 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:25.210192 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dab965a8b2544638322510c199897b6', 'request_id': 'req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.220033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dab965a8b2544638322510c199897b6', 'request_id': 'req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.227852 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dab965a8b2544638322510c199897b6', 'request_id': 'req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.238922 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7 admin admin] The token cache is being invalidate because project 8dab965a8b2544638322510c199897b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:25.239206 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.239849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.241348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1028/2088] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:25 2022] DELETE /identity/v3/projects/8dab965a8b2544638322510c199897b6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:25.245266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49618886-0b14-43dd-8e7c-31f2790c86ab service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:25.255321 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49618886-0b14-43dd-8e7c-31f2790c86ab service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:25.311752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58fa0b-b65e-4005-a3d4-734e5bbf51ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:25.312395 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58fa0b-b65e-4005-a3d4-734e5bbf51ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:25.312677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58fa0b-b65e-4005-a3d4-734e5bbf51ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:25.312950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58fa0b-b65e-4005-a3d4-734e5bbf51ef admin admin] PATH_INFO: `/v3/users/7b5b5254fbb7451a9c5c66f24446aacc` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:25.314112 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58fa0b-b65e-4005-a3d4-734e5bbf51ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b5b5254fbb7451a9c5c66f24446aacc)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:25.316059 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58fa0b-b65e-4005-a3d4-734e5bbf51ef admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:25.361479 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-6e58fa0b-b65e-4005-a3d4-734e5bbf51ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b5b5254fbb7451a9c5c66f24446aacc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.368285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-6e58fa0b-b65e-4005-a3d4-734e5bbf51ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b5b5254fbb7451a9c5c66f24446aacc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.379818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-6e58fa0b-b65e-4005-a3d4-734e5bbf51ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b5b5254fbb7451a9c5c66f24446aacc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.380480 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-6e58fa0b-b65e-4005-a3d4-734e5bbf51ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b5b5254fbb7451a9c5c66f24446aacc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.381082 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-6e58fa0b-b65e-4005-a3d4-734e5bbf51ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b5b5254fbb7451a9c5c66f24446aacc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.393995 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1061/2089] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:25 2022] DELETE /identity/v3/users/7b5b5254fbb7451a9c5c66f24446aacc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:25.453540 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:25.455082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:25.469154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bbab9-51cf-443d-b1ec-e1d7918a1fc7 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:25.534369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e55b4b-61db-457c-bf62-68e18f76dc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:25.534369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e55b4b-61db-457c-bf62-68e18f76dc94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:25.534369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e55b4b-61db-457c-bf62-68e18f76dc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:25.534369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e55b4b-61db-457c-bf62-68e18f76dc94 admin admin] PATH_INFO: `/v3/users/272840ea732c4291ba561e033796f301` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:25.535658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e55b4b-61db-457c-bf62-68e18f76dc94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=272840ea732c4291ba561e033796f301)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:25.538545 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e55b4b-61db-457c-bf62-68e18f76dc94 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:25.592202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c6e55b4b-61db-457c-bf62-68e18f76dc94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '272840ea732c4291ba561e033796f301'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.604412 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c6e55b4b-61db-457c-bf62-68e18f76dc94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '272840ea732c4291ba561e033796f301'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.613019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c6e55b4b-61db-457c-bf62-68e18f76dc94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '272840ea732c4291ba561e033796f301'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.613624 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c6e55b4b-61db-457c-bf62-68e18f76dc94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '272840ea732c4291ba561e033796f301'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.621082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c6e55b4b-61db-457c-bf62-68e18f76dc94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '272840ea732c4291ba561e033796f301'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.628019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1029/2090] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:25 2022] DELETE /identity/v3/users/272840ea732c4291ba561e033796f301 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:25.633069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f69d77-d32a-4ba6-a1fa-df4d708f78d3 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:25.633533 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f69d77-d32a-4ba6-a1fa-df4d708f78d3 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:25.633752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f69d77-d32a-4ba6-a1fa-df4d708f78d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:25.672351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-27f69d77-d32a-4ba6-a1fa-df4d708f78d3 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:25.673491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-27f69d77-d32a-4ba6-a1fa-df4d708f78d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:25.730556 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:25.783146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1062/2091] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:25.789737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e55b4b-61db-457c-bf62-68e18f76dc94 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:25.791381 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c6e55b4b-61db-457c-bf62-68e18f76dc94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:25.797738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f69d77-d32a-4ba6-a1fa-df4d708f78d3 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:25.804598 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-27f69d77-d32a-4ba6-a1fa-df4d708f78d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:25.823149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f69d77-d32a-4ba6-a1fa-df4d708f78d3 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:25.830972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3123845-66d9-4fce-9195-8e30fc289910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:25.831640 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3123845-66d9-4fce-9195-8e30fc289910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:25.831924 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3123845-66d9-4fce-9195-8e30fc289910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:25.832200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3123845-66d9-4fce-9195-8e30fc289910 admin admin] PATH_INFO: `/v3/projects/c1761ae5e8b541499e595b0fc9bcd939` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:25.833398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3123845-66d9-4fce-9195-8e30fc289910 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1761ae5e8b541499e595b0fc9bcd939)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:25.835352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3123845-66d9-4fce-9195-8e30fc289910 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:25.847342 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:25.878378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e55b4b-61db-457c-bf62-68e18f76dc94 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:25.885656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea71fa-4930-41ca-850b-015516f3f8de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:25.886318 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea71fa-4930-41ca-850b-015516f3f8de service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:25.886703 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea71fa-4930-41ca-850b-015516f3f8de service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:25.887043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea71fa-4930-41ca-850b-015516f3f8de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:25.897580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea71fa-4930-41ca-850b-015516f3f8de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:25.899143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea71fa-4930-41ca-850b-015516f3f8de service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:25.900156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d3123845-66d9-4fce-9195-8e30fc289910 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1761ae5e8b541499e595b0fc9bcd939', 'request_id': 'req-d3123845-66d9-4fce-9195-8e30fc289910'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.909716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d3123845-66d9-4fce-9195-8e30fc289910 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1761ae5e8b541499e595b0fc9bcd939', 'request_id': 'req-d3123845-66d9-4fce-9195-8e30fc289910'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.911766 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1030/2092] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:25.917285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d3123845-66d9-4fce-9195-8e30fc289910 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1761ae5e8b541499e595b0fc9bcd939', 'request_id': 'req-d3123845-66d9-4fce-9195-8e30fc289910'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.934250 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d3123845-66d9-4fce-9195-8e30fc289910 admin admin] The token cache is being invalidate because project c1761ae5e8b541499e595b0fc9bcd939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:25.934844 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d3123845-66d9-4fce-9195-8e30fc289910 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.935768 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d3123845-66d9-4fce-9195-8e30fc289910 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:25.938561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1063/2093] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:25 2022] DELETE /identity/v3/projects/c1761ae5e8b541499e595b0fc9bcd939 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:25.944170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28829b0-8347-40d0-98fa-b254d2bb1af7 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:25.945072 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28829b0-8347-40d0-98fa-b254d2bb1af7 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:25.945513 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28829b0-8347-40d0-98fa-b254d2bb1af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:25.969656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-b28829b0-8347-40d0-98fa-b254d2bb1af7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:25.971377 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-b28829b0-8347-40d0-98fa-b254d2bb1af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:26.043216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1031/2094] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:54:26.052781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c776f0-4019-4f62-b6d4-e5a3b5fdaa3d None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:26.053134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c776f0-4019-4f62-b6d4-e5a3b5fdaa3d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:26.053417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c776f0-4019-4f62-b6d4-e5a3b5fdaa3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:26.075570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-51c776f0-4019-4f62-b6d4-e5a3b5fdaa3d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:26.076818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-51c776f0-4019-4f62-b6d4-e5a3b5fdaa3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:26.085302 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1064/2095] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:26.089735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28829b0-8347-40d0-98fa-b254d2bb1af7 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:26.099884 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28829b0-8347-40d0-98fa-b254d2bb1af7 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:26.104907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dca7d-ee5e-49c4-a9a9-ca9ab4444a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:26.105535 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dca7d-ee5e-49c4-a9a9-ca9ab4444a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:26.105816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dca7d-ee5e-49c4-a9a9-ca9ab4444a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:26.106133 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3dca7d-ee5e-49c4-a9a9-ca9ab4444a0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:26.107015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3dca7d-ee5e-49c4-a9a9-ca9ab4444a0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:54:26.107331 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3dca7d-ee5e-49c4-a9a9-ca9ab4444a0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:26.109189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3dca7d-ee5e-49c4-a9a9-ca9ab4444a0e admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:26.123018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1032/2096] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:54:26.128146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c776f0-4019-4f62-b6d4-e5a3b5fdaa3d None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:26.137794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c776f0-4019-4f62-b6d4-e5a3b5fdaa3d None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:26.143464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4936c40-2f8c-4ced-8d3f-cbc0c0c77042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:26.144079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4936c40-2f8c-4ced-8d3f-cbc0c0c77042 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:26.144386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4936c40-2f8c-4ced-8d3f-cbc0c0c77042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:26.144652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4936c40-2f8c-4ced-8d3f-cbc0c0c77042 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:26.145434 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4936c40-2f8c-4ced-8d3f-cbc0c0c77042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:26.147374 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4936c40-2f8c-4ced-8d3f-cbc0c0c77042 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:26.177922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1065/2097] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:26 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:54:26.179178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dca7d-ee5e-49c4-a9a9-ca9ab4444a0e admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:26.189348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3dca7d-ee5e-49c4-a9a9-ca9ab4444a0e admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:26.221401 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4936c40-2f8c-4ced-8d3f-cbc0c0c77042 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:26.223663 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c4936c40-2f8c-4ced-8d3f-cbc0c0c77042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:26.263164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a16541-ec9c-4028-99c8-411e18df8726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:26.263863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a16541-ec9c-4028-99c8-411e18df8726 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:26.264143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a16541-ec9c-4028-99c8-411e18df8726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:26.264413 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a16541-ec9c-4028-99c8-411e18df8726 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:26.265077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a16541-ec9c-4028-99c8-411e18df8726 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:26.266996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a16541-ec9c-4028-99c8-411e18df8726 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:26.297556 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1033/2098] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:26 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:54:26.301893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a16541-ec9c-4028-99c8-411e18df8726 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:26.318777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a16541-ec9c-4028-99c8-411e18df8726 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:26.326277 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4936c40-2f8c-4ced-8d3f-cbc0c0c77042 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:26.327446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b34449-b56b-4aae-8922-d8997cf60659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:26.328161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b34449-b56b-4aae-8922-d8997cf60659 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:26.328519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b34449-b56b-4aae-8922-d8997cf60659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:26.328872 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b34449-b56b-4aae-8922-d8997cf60659 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:26.329655 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b34449-b56b-4aae-8922-d8997cf60659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:26.331395 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b34449-b56b-4aae-8922-d8997cf60659 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:26.332173 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b62c04-7972-430e-b41b-d9c9f9f9b2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:26.333082 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b62c04-7972-430e-b41b-d9c9f9f9b2ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:26.333360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b62c04-7972-430e-b41b-d9c9f9f9b2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:26.333632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b62c04-7972-430e-b41b-d9c9f9f9b2ea admin admin] PATH_INFO: `/v3/projects/0de87c3f70a84e23838af32e7c546e88` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:26.335417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b62c04-7972-430e-b41b-d9c9f9f9b2ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=0de87c3f70a84e23838af32e7c546e88)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:26.337281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b62c04-7972-430e-b41b-d9c9f9f9b2ea admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:26.353612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1034/2099] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:54:26.357626 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b34449-b56b-4aae-8922-d8997cf60659 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:26.368758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b34449-b56b-4aae-8922-d8997cf60659 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:26.373841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d90a39-e232-4418-9995-ba3e04e03f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:26.374518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d90a39-e232-4418-9995-ba3e04e03f97 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:26.374839 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d90a39-e232-4418-9995-ba3e04e03f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:26.375125 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d90a39-e232-4418-9995-ba3e04e03f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:26.375858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d90a39-e232-4418-9995-ba3e04e03f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:26.377836 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:26.378023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d90a39-e232-4418-9995-ba3e04e03f97 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:26.389234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1035/2100] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:54:26.393359 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d90a39-e232-4418-9995-ba3e04e03f97 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:26.396514 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e4b62c04-7972-430e-b41b-d9c9f9f9b2ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0de87c3f70a84e23838af32e7c546e88', 'request_id': 'req-e4b62c04-7972-430e-b41b-d9c9f9f9b2ea'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:26.403565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d90a39-e232-4418-9995-ba3e04e03f97 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:26.405034 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e4b62c04-7972-430e-b41b-d9c9f9f9b2ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0de87c3f70a84e23838af32e7c546e88', 'request_id': 'req-e4b62c04-7972-430e-b41b-d9c9f9f9b2ea'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:26.412382 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e4b62c04-7972-430e-b41b-d9c9f9f9b2ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0de87c3f70a84e23838af32e7c546e88', 'request_id': 'req-e4b62c04-7972-430e-b41b-d9c9f9f9b2ea'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:26.425241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e4b62c04-7972-430e-b41b-d9c9f9f9b2ea admin admin] The token cache is being invalidate because project 0de87c3f70a84e23838af32e7c546e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:26.425530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e4b62c04-7972-430e-b41b-d9c9f9f9b2ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:26.426459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e4b62c04-7972-430e-b41b-d9c9f9f9b2ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:26.428061 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1066/2101] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:26 2022] DELETE /identity/v3/projects/0de87c3f70a84e23838af32e7c546e88 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:26.442795 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:26.476750 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7465265-e398-4ec3-b26c-854b9a36223d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:26.477454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7465265-e398-4ec3-b26c-854b9a36223d admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:26.477740 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7465265-e398-4ec3-b26c-854b9a36223d admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:26.478020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7465265-e398-4ec3-b26c-854b9a36223d admin admin] PATH_INFO: `/v3/projects/945c505bee5d49289f5c551636ab5fd5/users/78a74f00fa384d9a8dd5625645a25f65/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:26.498579 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7465265-e398-4ec3-b26c-854b9a36223d admin admin] RBAC: Authorizing `identity:create_grant(project_id=945c505bee5d49289f5c551636ab5fd5, user_id=78a74f00fa384d9a8dd5625645a25f65, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:26.501619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7465265-e398-4ec3-b26c-854b9a36223d admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:26.519440 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1036/2102] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:26 2022] PUT /identity/v3/projects/945c505bee5d49289f5c551636ab5fd5/users/78a74f00fa384d9a8dd5625645a25f65/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:26.534452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c078d2-c30d-46a3-b179-8e6828baf18d None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:26.535053 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c078d2-c30d-46a3-b179-8e6828baf18d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:26.535415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c078d2-c30d-46a3-b179-8e6828baf18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:26.579870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-83c078d2-c30d-46a3-b179-8e6828baf18d None None] MFA Rules not processed for user `78a74f00fa384d9a8dd5625645a25f65`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:26.581694 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-83c078d2-c30d-46a3-b179-8e6828baf18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:26.658955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1067/2103] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:26.670283 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7465265-e398-4ec3-b26c-854b9a36223d admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:26.672050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a7465265-e398-4ec3-b26c-854b9a36223d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:26.731420 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7465265-e398-4ec3-b26c-854b9a36223d admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:26.735207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8343ae5-0c19-4683-b915-73a47dcb9cfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:26.735844 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8343ae5-0c19-4683-b915-73a47dcb9cfe service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:26.738281 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8343ae5-0c19-4683-b915-73a47dcb9cfe service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:26.738609 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8343ae5-0c19-4683-b915-73a47dcb9cfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:26.747979 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8343ae5-0c19-4683-b915-73a47dcb9cfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:26.749425 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8343ae5-0c19-4683-b915-73a47dcb9cfe service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:26.773321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1037/2104] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:34.082400 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203ff1f-15d6-44d5-a78a-c6ff55102e4f None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:34.082918 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203ff1f-15d6-44d5-a78a-c6ff55102e4f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:34.083150 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203ff1f-15d6-44d5-a78a-c6ff55102e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:34.102094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-4203ff1f-15d6-44d5-a78a-c6ff55102e4f None None] MFA Rules not processed for user `0e8dbb33c0494f01b5066affa7aa74c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:34.103262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-4203ff1f-15d6-44d5-a78a-c6ff55102e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:34.155874 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1068/2105] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:36.755361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=47,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:43.193261 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd9973-b2c2-4379-b5df-498f6ec44585 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:43.193862 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd9973-b2c2-4379-b5df-498f6ec44585 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:43.193862 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd9973-b2c2-4379-b5df-498f6ec44585 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:43.195467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1038/2106] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 3 12:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:54:43.206431 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3706e11-be1e-4a43-bf54-10c5cdd18057 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:43.207010 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3706e11-be1e-4a43-bf54-10c5cdd18057 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:43.207284 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3706e11-be1e-4a43-bf54-10c5cdd18057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:43.229447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-e3706e11-be1e-4a43-bf54-10c5cdd18057 None None] MFA Rules not processed for user `5094263ac471481db48783b19192e01d`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:43.231027 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e3706e11-be1e-4a43-bf54-10c5cdd18057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:43.293069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd9973-b2c2-4379-b5df-498f6ec44585 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:43.295199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-7efd9973-b2c2-4379-b5df-498f6ec44585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:43.301313 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:43.326811 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1069/2107] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 3 12:54:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:43.341756 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3706e11-be1e-4a43-bf54-10c5cdd18057 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:43.343865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e3706e11-be1e-4a43-bf54-10c5cdd18057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:43.364369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3706e11-be1e-4a43-bf54-10c5cdd18057 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:43.370321 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8597d-8ad5-4d14-b8e3-8ac3e6fd4865 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5094263ac471481db48783b19192e01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:43.370612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd9973-b2c2-4379-b5df-498f6ec44585 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:43.371335 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8597d-8ad5-4d14-b8e3-8ac3e6fd4865 service placement] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:43.371809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8597d-8ad5-4d14-b8e3-8ac3e6fd4865 service placement] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:43.372227 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8597d-8ad5-4d14-b8e3-8ac3e6fd4865 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:43.375292 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9eb2f7-7523-457a-891a-ec5dce90c393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:43.375929 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9eb2f7-7523-457a-891a-ec5dce90c393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:43.376213 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9eb2f7-7523-457a-891a-ec5dce90c393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:43.376486 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9eb2f7-7523-457a-891a-ec5dce90c393 admin admin] PATH_INFO: `/v3/users/8aad259ca09340fb99c6b81b4f096dc5` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:43.378041 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9eb2f7-7523-457a-891a-ec5dce90c393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aad259ca09340fb99c6b81b4f096dc5)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:43.379923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9eb2f7-7523-457a-891a-ec5dce90c393 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:43.383512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8597d-8ad5-4d14-b8e3-8ac3e6fd4865 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:43.385097 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8597d-8ad5-4d14-b8e3-8ac3e6fd4865 service placement] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:43.396891 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1070/2108] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:54:43.440053 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ee9eb2f7-7523-457a-891a-ec5dce90c393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aad259ca09340fb99c6b81b4f096dc5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:43.453275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ee9eb2f7-7523-457a-891a-ec5dce90c393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aad259ca09340fb99c6b81b4f096dc5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:43.461302 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ee9eb2f7-7523-457a-891a-ec5dce90c393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aad259ca09340fb99c6b81b4f096dc5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:43.462272 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ee9eb2f7-7523-457a-891a-ec5dce90c393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aad259ca09340fb99c6b81b4f096dc5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:43.469328 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ee9eb2f7-7523-457a-891a-ec5dce90c393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aad259ca09340fb99c6b81b4f096dc5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:43.476103 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1039/2109] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:43 2022] DELETE /identity/v3/users/8aad259ca09340fb99c6b81b4f096dc5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:43.650273 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8597d-8ad5-4d14-b8e3-8ac3e6fd4865 service placement] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:43.670087 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8597d-8ad5-4d14-b8e3-8ac3e6fd4865 service placement] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:43.749581 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988f178-e12d-4765-bbb6-a863189c3175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:43.750226 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988f178-e12d-4765-bbb6-a863189c3175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:43.750524 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988f178-e12d-4765-bbb6-a863189c3175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:43.750799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988f178-e12d-4765-bbb6-a863189c3175 admin admin] PATH_INFO: `/v3/projects/6f5ed78b61e1440da93d5386d4d329f9` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:43.751950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988f178-e12d-4765-bbb6-a863189c3175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f5ed78b61e1440da93d5386d4d329f9)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:43.753960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2988f178-e12d-4765-bbb6-a863189c3175 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:43.800655 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2988f178-e12d-4765-bbb6-a863189c3175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f5ed78b61e1440da93d5386d4d329f9', 'request_id': 'req-2988f178-e12d-4765-bbb6-a863189c3175'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:43.811935 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2988f178-e12d-4765-bbb6-a863189c3175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f5ed78b61e1440da93d5386d4d329f9', 'request_id': 'req-2988f178-e12d-4765-bbb6-a863189c3175'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:43.819496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2988f178-e12d-4765-bbb6-a863189c3175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f5ed78b61e1440da93d5386d4d329f9', 'request_id': 'req-2988f178-e12d-4765-bbb6-a863189c3175'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:43.831642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2988f178-e12d-4765-bbb6-a863189c3175 admin admin] The token cache is being invalidate because project 6f5ed78b61e1440da93d5386d4d329f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:43.831999 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2988f178-e12d-4765-bbb6-a863189c3175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:43.833347 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2988f178-e12d-4765-bbb6-a863189c3175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:43.835220 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1071/2110] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:43 2022] DELETE /identity/v3/projects/6f5ed78b61e1440da93d5386d4d329f9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:43.839988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb471c-e407-4be0-a469-84d3a9276e47 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:43.840378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb471c-e407-4be0-a469-84d3a9276e47 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:43.840748 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb471c-e407-4be0-a469-84d3a9276e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:43.863038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-2acb471c-e407-4be0-a469-84d3a9276e47 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:43.864200 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2acb471c-e407-4be0-a469-84d3a9276e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:43.924982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1040/2111] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:43.933274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6ecad-9bfc-428b-a872-4d6750d6d808 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:43.933565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6ecad-9bfc-428b-a872-4d6750d6d808 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:43.933843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c6ecad-9bfc-428b-a872-4d6750d6d808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:43.953986 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-b0c6ecad-9bfc-428b-a872-4d6750d6d808 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:43.955228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b0c6ecad-9bfc-428b-a872-4d6750d6d808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:43.963778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1072/2112] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:43.968106 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb471c-e407-4be0-a469-84d3a9276e47 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:43.977867 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb471c-e407-4be0-a469-84d3a9276e47 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:43.982465 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e3714-fddc-461e-95b6-09e6b5939b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:43.983099 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e3714-fddc-461e-95b6-09e6b5939b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:43.983376 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e3714-fddc-461e-95b6-09e6b5939b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:43.983649 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589e3714-fddc-461e-95b6-09e6b5939b60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:43.984340 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e3714-fddc-461e-95b6-09e6b5939b60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:54:43.984643 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e3714-fddc-461e-95b6-09e6b5939b60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:43.986220 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589e3714-fddc-461e-95b6-09e6b5939b60 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:43.998127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1041/2113] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:54:44.002639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6ecad-9bfc-428b-a872-4d6750d6d808 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:44.012777 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c6ecad-9bfc-428b-a872-4d6750d6d808 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:44.017544 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f0d82-9d18-430c-9ff8-0696292b4d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:44.018154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f0d82-9d18-430c-9ff8-0696292b4d24 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:44.018437 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f0d82-9d18-430c-9ff8-0696292b4d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:44.018736 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f0d82-9d18-430c-9ff8-0696292b4d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:44.019528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f0d82-9d18-430c-9ff8-0696292b4d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:44.021479 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f0d82-9d18-430c-9ff8-0696292b4d24 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:44.052471 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1073/2114] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:44 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:44.056691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e3714-fddc-461e-95b6-09e6b5939b60 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:44.067886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589e3714-fddc-461e-95b6-09e6b5939b60 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:44.071759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:44.126450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c7af2-3136-4372-b4c6-c07631ef6b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:44.127094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c7af2-3136-4372-b4c6-c07631ef6b90 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:44.127374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c7af2-3136-4372-b4c6-c07631ef6b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:44.127645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c7af2-3136-4372-b4c6-c07631ef6b90 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:44.128304 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c7af2-3136-4372-b4c6-c07631ef6b90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:44.130215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c7af2-3136-4372-b4c6-c07631ef6b90 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:44.152233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1042/2115] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:44 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:54:44.156332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f0d82-9d18-430c-9ff8-0696292b4d24 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:44.167199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f0d82-9d18-430c-9ff8-0696292b4d24 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:44.171864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa50f8d-266a-4a02-9f4d-4620478c82c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:44.172544 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa50f8d-266a-4a02-9f4d-4620478c82c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:44.172892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa50f8d-266a-4a02-9f4d-4620478c82c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:44.173237 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa50f8d-266a-4a02-9f4d-4620478c82c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:44.174000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa50f8d-266a-4a02-9f4d-4620478c82c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:44.176186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa50f8d-266a-4a02-9f4d-4620478c82c7 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:44.186361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1074/2116] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:44.190367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c7af2-3136-4372-b4c6-c07631ef6b90 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:44.201152 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c7af2-3136-4372-b4c6-c07631ef6b90 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:44.205949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2d138-5e7e-4979-aacc-5175a444d26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:44.206629 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2d138-5e7e-4979-aacc-5175a444d26b admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:44.207042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2d138-5e7e-4979-aacc-5175a444d26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:44.207782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2d138-5e7e-4979-aacc-5175a444d26b admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:44.208471 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2d138-5e7e-4979-aacc-5175a444d26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:44.210057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2d138-5e7e-4979-aacc-5175a444d26b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:44.225367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1043/2117] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:54:44.226365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa50f8d-266a-4a02-9f4d-4620478c82c7 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:44.237132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa50f8d-266a-4a02-9f4d-4620478c82c7 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:44.242020 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57f838-fda5-4af4-8e7f-27216eb26501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:44.242750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57f838-fda5-4af4-8e7f-27216eb26501 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:44.243117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57f838-fda5-4af4-8e7f-27216eb26501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:44.243293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57f838-fda5-4af4-8e7f-27216eb26501 admin admin] PATH_INFO: `/v3/projects/48b17715e6f74eb9ac5bbcbf8b8b16cc/users/92c5efe98f2d4588b9a543f2c000f261/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:44.261781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57f838-fda5-4af4-8e7f-27216eb26501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48b17715e6f74eb9ac5bbcbf8b8b16cc, user_id=92c5efe98f2d4588b9a543f2c000f261, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:44.264470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57f838-fda5-4af4-8e7f-27216eb26501 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:44.274608 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:44.279763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1075/2118] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:44 2022] PUT /identity/v3/projects/48b17715e6f74eb9ac5bbcbf8b8b16cc/users/92c5efe98f2d4588b9a543f2c000f261/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:44.292498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d58df6-b17c-4540-b4e1-7a806ad4cb70 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:44.292668 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d58df6-b17c-4540-b4e1-7a806ad4cb70 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:44.292935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d58df6-b17c-4540-b4e1-7a806ad4cb70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:44.322865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-75d58df6-b17c-4540-b4e1-7a806ad4cb70 None None] MFA Rules not processed for user `92c5efe98f2d4588b9a543f2c000f261`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:44.323843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-75d58df6-b17c-4540-b4e1-7a806ad4cb70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:44.413410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1044/2119] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:44.424378 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57f838-fda5-4af4-8e7f-27216eb26501 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:44.426382 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-2c57f838-fda5-4af4-8e7f-27216eb26501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:44.507661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57f838-fda5-4af4-8e7f-27216eb26501 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:44.513832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98064c6d-573d-4f5c-8d4a-a70b3b678a4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:44.514653 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98064c6d-573d-4f5c-8d4a-a70b3b678a4c service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:44.515097 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98064c6d-573d-4f5c-8d4a-a70b3b678a4c service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:44.515510 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98064c6d-573d-4f5c-8d4a-a70b3b678a4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:44.525495 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98064c6d-573d-4f5c-8d4a-a70b3b678a4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:44.527172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98064c6d-573d-4f5c-8d4a-a70b3b678a4c service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:44.539148 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1076/2120] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:47.221639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d58df6-b17c-4540-b4e1-7a806ad4cb70 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:47.283072 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d58df6-b17c-4540-b4e1-7a806ad4cb70 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:47.288489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f10412-f33e-40ac-9738-0042e637bff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:47.289113 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f10412-f33e-40ac-9738-0042e637bff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:47.289388 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f10412-f33e-40ac-9738-0042e637bff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:47.289659 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f10412-f33e-40ac-9738-0042e637bff2 admin admin] PATH_INFO: `/v3/users/92c5efe98f2d4588b9a543f2c000f261` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:47.290847 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f10412-f33e-40ac-9738-0042e637bff2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92c5efe98f2d4588b9a543f2c000f261)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:47.292774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f10412-f33e-40ac-9738-0042e637bff2 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:47.308711 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:47.339290 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-82f10412-f33e-40ac-9738-0042e637bff2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92c5efe98f2d4588b9a543f2c000f261'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:47.352336 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-82f10412-f33e-40ac-9738-0042e637bff2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92c5efe98f2d4588b9a543f2c000f261'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:47.360026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-82f10412-f33e-40ac-9738-0042e637bff2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92c5efe98f2d4588b9a543f2c000f261'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:47.360726 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-82f10412-f33e-40ac-9738-0042e637bff2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92c5efe98f2d4588b9a543f2c000f261'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:47.367820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-82f10412-f33e-40ac-9738-0042e637bff2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92c5efe98f2d4588b9a543f2c000f261'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:47.373734 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1045/2121] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:47 2022] DELETE /identity/v3/users/92c5efe98f2d4588b9a543f2c000f261 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:47.377966 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2cd192-ac3a-4421-9967-b84c173e374a None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:47.378446 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2cd192-ac3a-4421-9967-b84c173e374a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:47.378607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2cd192-ac3a-4421-9967-b84c173e374a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:47.398786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-0c2cd192-ac3a-4421-9967-b84c173e374a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:47.400007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0c2cd192-ac3a-4421-9967-b84c173e374a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:47.459542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1077/2122] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:47.469163 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f10412-f33e-40ac-9738-0042e637bff2 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:47.470626 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-82f10412-f33e-40ac-9738-0042e637bff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:47.531262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f10412-f33e-40ac-9738-0042e637bff2 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:47.537773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf7a6e-c2f8-4419-8d10-06093e90d5e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:47.539104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf7a6e-c2f8-4419-8d10-06093e90d5e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:47.539537 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf7a6e-c2f8-4419-8d10-06093e90d5e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:47.539825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf7a6e-c2f8-4419-8d10-06093e90d5e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:47.551458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf7a6e-c2f8-4419-8d10-06093e90d5e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:47.552891 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf7a6e-c2f8-4419-8d10-06093e90d5e4 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:47.565244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1046/2123] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:47.872025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2cd192-ac3a-4421-9967-b84c173e374a None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:47.877999 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0c2cd192-ac3a-4421-9967-b84c173e374a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:47.886460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2cd192-ac3a-4421-9967-b84c173e374a None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:47.891054 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe776c8-e72a-4bcf-88e7-981e0189979f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:47.891696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe776c8-e72a-4bcf-88e7-981e0189979f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:47.891978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe776c8-e72a-4bcf-88e7-981e0189979f admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:47.892253 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe776c8-e72a-4bcf-88e7-981e0189979f admin admin] PATH_INFO: `/v3/projects/48b17715e6f74eb9ac5bbcbf8b8b16cc` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:47.893418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe776c8-e72a-4bcf-88e7-981e0189979f admin admin] RBAC: Authorizing `identity:delete_project(project_id=48b17715e6f74eb9ac5bbcbf8b8b16cc)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:47.895386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe776c8-e72a-4bcf-88e7-981e0189979f admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:47.947907 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-cfe776c8-e72a-4bcf-88e7-981e0189979f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48b17715e6f74eb9ac5bbcbf8b8b16cc', 'request_id': 'req-cfe776c8-e72a-4bcf-88e7-981e0189979f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:47.954681 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-cfe776c8-e72a-4bcf-88e7-981e0189979f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48b17715e6f74eb9ac5bbcbf8b8b16cc', 'request_id': 'req-cfe776c8-e72a-4bcf-88e7-981e0189979f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:47.960961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-cfe776c8-e72a-4bcf-88e7-981e0189979f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48b17715e6f74eb9ac5bbcbf8b8b16cc', 'request_id': 'req-cfe776c8-e72a-4bcf-88e7-981e0189979f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:47.972305 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-cfe776c8-e72a-4bcf-88e7-981e0189979f admin admin] The token cache is being invalidate because project 48b17715e6f74eb9ac5bbcbf8b8b16cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:47.972538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-cfe776c8-e72a-4bcf-88e7-981e0189979f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:47.973197 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-cfe776c8-e72a-4bcf-88e7-981e0189979f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:47.974741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1078/2124] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:47 2022] DELETE /identity/v3/projects/48b17715e6f74eb9ac5bbcbf8b8b16cc => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:47.979527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bec1a-7fe1-4f0f-9a03-0a9a039cf871 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:47.979816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bec1a-7fe1-4f0f-9a03-0a9a039cf871 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:47.980091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bec1a-7fe1-4f0f-9a03-0a9a039cf871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.000667 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-929bec1a-7fe1-4f0f-9a03-0a9a039cf871 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:48.002300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-929bec1a-7fe1-4f0f-9a03-0a9a039cf871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:48.072142 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1047/2125] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:48.083012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c396a-0547-4633-bdd4-94e251b81da4 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:48.083012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c396a-0547-4633-bdd4-94e251b81da4 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:48.083012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c396a-0547-4633-bdd4-94e251b81da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.101673 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-980c396a-0547-4633-bdd4-94e251b81da4 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:48.102897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-980c396a-0547-4633-bdd4-94e251b81da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:48.111291 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1079/2126] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:48.116012 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bec1a-7fe1-4f0f-9a03-0a9a039cf871 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:48.126206 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bec1a-7fe1-4f0f-9a03-0a9a039cf871 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:48.131131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db0c96-66d2-4086-90b5-516cb479afeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:48.131886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db0c96-66d2-4086-90b5-516cb479afeb admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:48.132237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db0c96-66d2-4086-90b5-516cb479afeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:48.132578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db0c96-66d2-4086-90b5-516cb479afeb admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.133348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db0c96-66d2-4086-90b5-516cb479afeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:54:48.133723 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db0c96-66d2-4086-90b5-516cb479afeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:48.135437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db0c96-66d2-4086-90b5-516cb479afeb admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:48.147854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1048/2127] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:54:48.152569 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c396a-0547-4633-bdd4-94e251b81da4 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:48.157059 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:48.164836 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c396a-0547-4633-bdd4-94e251b81da4 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:48.167078 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db0c96-66d2-4086-90b5-516cb479afeb admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:48.168762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-27db0c96-66d2-4086-90b5-516cb479afeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:48.171288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da45e0-d354-4b81-8ed2-ab161c612ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:48.171908 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da45e0-d354-4b81-8ed2-ab161c612ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:48.172215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da45e0-d354-4b81-8ed2-ab161c612ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:48.172491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da45e0-d354-4b81-8ed2-ab161c612ab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.173159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da45e0-d354-4b81-8ed2-ab161c612ab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:48.175087 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57da45e0-d354-4b81-8ed2-ab161c612ab7 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:48.182516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db0c96-66d2-4086-90b5-516cb479afeb admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:48.187357 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cba965-2e3b-4d4d-9f01-23e6920aca40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:48.187995 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cba965-2e3b-4d4d-9f01-23e6920aca40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:48.188280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cba965-2e3b-4d4d-9f01-23e6920aca40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:48.188557 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cba965-2e3b-4d4d-9f01-23e6920aca40 admin admin] PATH_INFO: `/v3/users/78a74f00fa384d9a8dd5625645a25f65` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.189750 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cba965-2e3b-4d4d-9f01-23e6920aca40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a74f00fa384d9a8dd5625645a25f65)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:48.191737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cba965-2e3b-4d4d-9f01-23e6920aca40 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:48.204004 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1080/2128] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:48 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:48.209286 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da45e0-d354-4b81-8ed2-ab161c612ab7 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:48.213097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:48.242324 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-36cba965-2e3b-4d4d-9f01-23e6920aca40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a74f00fa384d9a8dd5625645a25f65'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:48.254234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-36cba965-2e3b-4d4d-9f01-23e6920aca40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a74f00fa384d9a8dd5625645a25f65'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:48.261808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-36cba965-2e3b-4d4d-9f01-23e6920aca40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a74f00fa384d9a8dd5625645a25f65'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:48.262571 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-36cba965-2e3b-4d4d-9f01-23e6920aca40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a74f00fa384d9a8dd5625645a25f65'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:48.270444 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-36cba965-2e3b-4d4d-9f01-23e6920aca40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a74f00fa384d9a8dd5625645a25f65'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:48.275358 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da45e0-d354-4b81-8ed2-ab161c612ab7 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:48.277030 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1049/2129] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:48 2022] DELETE /identity/v3/users/78a74f00fa384d9a8dd5625645a25f65 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:48.281226 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05986c-790e-451a-89ce-2abd89981646 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:48.281656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05986c-790e-451a-89ce-2abd89981646 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:48.281945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05986c-790e-451a-89ce-2abd89981646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.306864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-cb05986c-790e-451a-89ce-2abd89981646 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:48.308283 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-cb05986c-790e-451a-89ce-2abd89981646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:48.336825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9a7b2-4f1e-4f60-a042-21a0b9f4e805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:48.337443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9a7b2-4f1e-4f60-a042-21a0b9f4e805 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:48.337722 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9a7b2-4f1e-4f60-a042-21a0b9f4e805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:48.337998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd9a7b2-4f1e-4f60-a042-21a0b9f4e805 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.338691 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9a7b2-4f1e-4f60-a042-21a0b9f4e805 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:48.340620 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd9a7b2-4f1e-4f60-a042-21a0b9f4e805 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:48.362532 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1081/2130] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:48 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:48.366562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9a7b2-4f1e-4f60-a042-21a0b9f4e805 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:48.368160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5bd9a7b2-4f1e-4f60-a042-21a0b9f4e805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:48.369648 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1050/2131] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:54:48.380049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05986c-790e-451a-89ce-2abd89981646 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:48.380212 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd9a7b2-4f1e-4f60-a042-21a0b9f4e805 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:48.382130 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-cb05986c-790e-451a-89ce-2abd89981646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:48.385210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6d682-86a6-41b5-990b-28e356a9e32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:48.385957 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6d682-86a6-41b5-990b-28e356a9e32a admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:48.386362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6d682-86a6-41b5-990b-28e356a9e32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:48.386609 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc6d682-86a6-41b5-990b-28e356a9e32a admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.387309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6d682-86a6-41b5-990b-28e356a9e32a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:48.388898 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc6d682-86a6-41b5-990b-28e356a9e32a admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:48.399194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1082/2132] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:54:48.403162 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6d682-86a6-41b5-990b-28e356a9e32a admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:48.413348 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc6d682-86a6-41b5-990b-28e356a9e32a admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:48.418047 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf235db-5226-4f4e-9d97-487f1eb23901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:48.418671 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf235db-5226-4f4e-9d97-487f1eb23901 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:48.418950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf235db-5226-4f4e-9d97-487f1eb23901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:48.419230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf235db-5226-4f4e-9d97-487f1eb23901 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.419932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf235db-5226-4f4e-9d97-487f1eb23901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:48.421539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf235db-5226-4f4e-9d97-487f1eb23901 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:48.433084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1083/2133] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:48.437320 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf235db-5226-4f4e-9d97-487f1eb23901 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:48.443149 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05986c-790e-451a-89ce-2abd89981646 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:48.446820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7043f-b8d2-4be5-8eac-87f83c3de16d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:48.447462 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf235db-5226-4f4e-9d97-487f1eb23901 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:48.447580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7043f-b8d2-4be5-8eac-87f83c3de16d service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:48.447747 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7043f-b8d2-4be5-8eac-87f83c3de16d service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:48.448073 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7043f-b8d2-4be5-8eac-87f83c3de16d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.451999 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f826bc-c260-4b5c-a320-6b4caae8b9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:48.452634 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f826bc-c260-4b5c-a320-6b4caae8b9bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:48.452914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f826bc-c260-4b5c-a320-6b4caae8b9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:48.453187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f826bc-c260-4b5c-a320-6b4caae8b9bf admin admin] PATH_INFO: `/v3/projects/da55b2e081da4144aca117dbcd03d846/users/29d2c32e972d49e0825c035771edf469/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.458778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7043f-b8d2-4be5-8eac-87f83c3de16d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:48.460511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7043f-b8d2-4be5-8eac-87f83c3de16d service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:48.468345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f826bc-c260-4b5c-a320-6b4caae8b9bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=da55b2e081da4144aca117dbcd03d846, user_id=29d2c32e972d49e0825c035771edf469, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:48.471013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f826bc-c260-4b5c-a320-6b4caae8b9bf admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:48.471328 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1051/2134] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:48.485040 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1084/2135] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:48 2022] PUT /identity/v3/projects/da55b2e081da4144aca117dbcd03d846/users/29d2c32e972d49e0825c035771edf469/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:48.496108 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ff0b0-edbf-4469-ace3-6af9787fbe8e None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:48.496325 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ff0b0-edbf-4469-ace3-6af9787fbe8e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:48.496597 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ff0b0-edbf-4469-ace3-6af9787fbe8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.527184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-ab5ff0b0-edbf-4469-ace3-6af9787fbe8e None None] MFA Rules not processed for user `29d2c32e972d49e0825c035771edf469`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:48.528417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ab5ff0b0-edbf-4469-ace3-6af9787fbe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:48.559600 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:48.599362 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1052/2136] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:48.603784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f826bc-c260-4b5c-a320-6b4caae8b9bf admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:48.648117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:48.684300 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f826bc-c260-4b5c-a320-6b4caae8b9bf admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:48.689611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377616e-5101-407e-93fd-193292901055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:48.690321 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377616e-5101-407e-93fd-193292901055 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:48.690753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377616e-5101-407e-93fd-193292901055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:48.691097 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d377616e-5101-407e-93fd-193292901055 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.692048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377616e-5101-407e-93fd-193292901055 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:48.694228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d377616e-5101-407e-93fd-193292901055 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:48.726785 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1085/2137] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:48 2022] POST /identity/v3/projects => generated 387 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:48.731224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ff0b0-edbf-4469-ace3-6af9787fbe8e None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:48.742019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ff0b0-edbf-4469-ace3-6af9787fbe8e None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:48.804018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6fa71-dad4-4bb5-88ef-b1ddcb3bcfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:48.804753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6fa71-dad4-4bb5-88ef-b1ddcb3bcfb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:48.805209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6fa71-dad4-4bb5-88ef-b1ddcb3bcfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:48.805562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6fa71-dad4-4bb5-88ef-b1ddcb3bcfb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.806189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6fa71-dad4-4bb5-88ef-b1ddcb3bcfb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:48.808187 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6fa71-dad4-4bb5-88ef-b1ddcb3bcfb3 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:48.828881 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377616e-5101-407e-93fd-193292901055 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:48.829981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1053/2138] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:48 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:48.834447 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6fa71-dad4-4bb5-88ef-b1ddcb3bcfb3 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:48.835529 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d377616e-5101-407e-93fd-193292901055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:48.845367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6fa71-dad4-4bb5-88ef-b1ddcb3bcfb3 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:48.852309 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d657f-f6da-4946-8748-fc6777a0e280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:48.853326 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d657f-f6da-4946-8748-fc6777a0e280 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:48.853732 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d657f-f6da-4946-8748-fc6777a0e280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:48.854145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d657f-f6da-4946-8748-fc6777a0e280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.855224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d657f-f6da-4946-8748-fc6777a0e280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:48.857952 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d657f-f6da-4946-8748-fc6777a0e280 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:48.873322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1054/2139] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:54:48.877341 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d377616e-5101-407e-93fd-193292901055 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:48.877879 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d657f-f6da-4946-8748-fc6777a0e280 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:48.889161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d657f-f6da-4946-8748-fc6777a0e280 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:48.896797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514146ee-ddeb-4ffa-8480-bf668434da26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:48.897565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514146ee-ddeb-4ffa-8480-bf668434da26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:48.897885 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514146ee-ddeb-4ffa-8480-bf668434da26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:48.898288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514146ee-ddeb-4ffa-8480-bf668434da26 admin admin] PATH_INFO: `/v3/projects/945c505bee5d49289f5c551636ab5fd5` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.899856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514146ee-ddeb-4ffa-8480-bf668434da26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=945c505bee5d49289f5c551636ab5fd5)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:48.900592 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e616cf-926b-4081-a20a-27d3caa71827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:48.901348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e616cf-926b-4081-a20a-27d3caa71827 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:48.901766 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e616cf-926b-4081-a20a-27d3caa71827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:48.902111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e616cf-926b-4081-a20a-27d3caa71827 admin admin] PATH_INFO: `/v3/projects/f541cf4b4a68421a81187d858e3173f8/users/98b24dfaf3454f15b9366f5c7a4c7dbc/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:48.902226 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514146ee-ddeb-4ffa-8480-bf668434da26 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:48.921873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e616cf-926b-4081-a20a-27d3caa71827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f541cf4b4a68421a81187d858e3173f8, user_id=98b24dfaf3454f15b9366f5c7a4c7dbc, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:48.924606 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e616cf-926b-4081-a20a-27d3caa71827 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:48.942536 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1055/2140] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:48 2022] PUT /identity/v3/projects/f541cf4b4a68421a81187d858e3173f8/users/98b24dfaf3454f15b9366f5c7a4c7dbc/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:48.946237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e616cf-926b-4081-a20a-27d3caa71827 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:48.961942 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-514146ee-ddeb-4ffa-8480-bf668434da26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '945c505bee5d49289f5c551636ab5fd5', 'request_id': 'req-514146ee-ddeb-4ffa-8480-bf668434da26'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:48.972864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-514146ee-ddeb-4ffa-8480-bf668434da26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '945c505bee5d49289f5c551636ab5fd5', 'request_id': 'req-514146ee-ddeb-4ffa-8480-bf668434da26'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:48.979961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-514146ee-ddeb-4ffa-8480-bf668434da26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '945c505bee5d49289f5c551636ab5fd5', 'request_id': 'req-514146ee-ddeb-4ffa-8480-bf668434da26'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:48.992704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-514146ee-ddeb-4ffa-8480-bf668434da26 admin admin] The token cache is being invalidate because project 945c505bee5d49289f5c551636ab5fd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:48.993096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-514146ee-ddeb-4ffa-8480-bf668434da26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:48.993860 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-514146ee-ddeb-4ffa-8480-bf668434da26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:48.995570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1086/2141] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:48 2022] DELETE /identity/v3/projects/945c505bee5d49289f5c551636ab5fd5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:49.002397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efa4f6-2117-469d-bbd8-6be8091c8e33 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.002662 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efa4f6-2117-469d-bbd8-6be8091c8e33 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.002945 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07efa4f6-2117-469d-bbd8-6be8091c8e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.020864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e616cf-926b-4081-a20a-27d3caa71827 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:49.024642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-07efa4f6-2117-469d-bbd8-6be8091c8e33 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:49.025869 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-07efa4f6-2117-469d-bbd8-6be8091c8e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:49.055587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:49.090110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fbadd-0626-4147-92d8-f675e3de768b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:49.090924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fbadd-0626-4147-92d8-f675e3de768b admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.091118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fbadd-0626-4147-92d8-f675e3de768b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.091390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fbadd-0626-4147-92d8-f675e3de768b admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.092161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fbadd-0626-4147-92d8-f675e3de768b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:49.093756 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fbadd-0626-4147-92d8-f675e3de768b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:49.107746 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1056/2142] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:54:49.110797 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fbadd-0626-4147-92d8-f675e3de768b admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:49.113301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d14fbadd-0626-4147-92d8-f675e3de768b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:49.117607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1087/2143] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:54:49.126022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87196e02-d175-4640-a2cd-749305319dfa None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.126449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87196e02-d175-4640-a2cd-749305319dfa None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.126766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87196e02-d175-4640-a2cd-749305319dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.127955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fbadd-0626-4147-92d8-f675e3de768b admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:49.134788 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33abd56-f351-4d1e-8b3c-b2e0210bfd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:49.135452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33abd56-f351-4d1e-8b3c-b2e0210bfd0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.135731 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33abd56-f351-4d1e-8b3c-b2e0210bfd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.136003 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33abd56-f351-4d1e-8b3c-b2e0210bfd0b admin admin] PATH_INFO: `/v3/projects/f541cf4b4a68421a81187d858e3173f8/users/98b24dfaf3454f15b9366f5c7a4c7dbc/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.138091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33abd56-f351-4d1e-8b3c-b2e0210bfd0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f541cf4b4a68421a81187d858e3173f8, user_id=98b24dfaf3454f15b9366f5c7a4c7dbc, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:49.140749 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33abd56-f351-4d1e-8b3c-b2e0210bfd0b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:49.149533 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-87196e02-d175-4640-a2cd-749305319dfa None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:49.150778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-87196e02-d175-4640-a2cd-749305319dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:49.153697 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1057/2144] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:49 2022] PUT /identity/v3/projects/f541cf4b4a68421a81187d858e3173f8/users/98b24dfaf3454f15b9366f5c7a4c7dbc/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:49.164508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475766ea-0aad-4590-ad03-a12cbcad2c65 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.164943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475766ea-0aad-4590-ad03-a12cbcad2c65 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.165342 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475766ea-0aad-4590-ad03-a12cbcad2c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.185811 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:49.195991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-475766ea-0aad-4590-ad03-a12cbcad2c65 None None] MFA Rules not processed for user `98b24dfaf3454f15b9366f5c7a4c7dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:49.197308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-475766ea-0aad-4590-ad03-a12cbcad2c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:49.215294 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1088/2145] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:54:49.219621 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87196e02-d175-4640-a2cd-749305319dfa None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:49.228455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87196e02-d175-4640-a2cd-749305319dfa None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:49.233103 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ca53c-6d22-4493-92ab-bf7d0b96ee84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:49.233696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ca53c-6d22-4493-92ab-bf7d0b96ee84 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.233992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ca53c-6d22-4493-92ab-bf7d0b96ee84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.234541 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ca53c-6d22-4493-92ab-bf7d0b96ee84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.235441 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ca53c-6d22-4493-92ab-bf7d0b96ee84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:54:49.235750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ca53c-6d22-4493-92ab-bf7d0b96ee84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:49.237377 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ca53c-6d22-4493-92ab-bf7d0b96ee84 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:49.249248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1089/2146] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:54:49.253555 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ca53c-6d22-4493-92ab-bf7d0b96ee84 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:49.263525 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ca53c-6d22-4493-92ab-bf7d0b96ee84 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:49.268377 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7c74e-ad3c-4721-bfca-ce65ae422ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:49.268961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7c74e-ad3c-4721-bfca-ce65ae422ee8 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.269242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7c74e-ad3c-4721-bfca-ce65ae422ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.269515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f7c74e-ad3c-4721-bfca-ce65ae422ee8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.270195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7c74e-ad3c-4721-bfca-ce65ae422ee8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:49.271645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1058/2147] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:54:49.275041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f7c74e-ad3c-4721-bfca-ce65ae422ee8 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:49.284129 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475766ea-0aad-4590-ad03-a12cbcad2c65 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:49.285549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-475766ea-0aad-4590-ad03-a12cbcad2c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:49.303399 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1090/2148] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:49 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:54:49.307762 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7c74e-ad3c-4721-bfca-ce65ae422ee8 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:49.346900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475766ea-0aad-4590-ad03-a12cbcad2c65 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:49.376551 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f7c74e-ad3c-4721-bfca-ce65ae422ee8 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:49.382053 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85176eb-1d66-463d-a020-3c840f2992c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:49.382713 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85176eb-1d66-463d-a020-3c840f2992c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.383000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85176eb-1d66-463d-a020-3c840f2992c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.383278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85176eb-1d66-463d-a020-3c840f2992c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.383951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85176eb-1d66-463d-a020-3c840f2992c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:49.385880 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85176eb-1d66-463d-a020-3c840f2992c0 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:49.407224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0766b-e080-4417-9426-ba634ef0e892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:49.407940 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0766b-e080-4417-9426-ba634ef0e892 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.408295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0766b-e080-4417-9426-ba634ef0e892 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.408792 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0766b-e080-4417-9426-ba634ef0e892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.414761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1091/2149] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:49 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:54:49.419301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85176eb-1d66-463d-a020-3c840f2992c0 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:49.421805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:49.433061 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85176eb-1d66-463d-a020-3c840f2992c0 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:49.437918 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ad01d-d080-40dc-a158-cb4c2dcfbca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:49.438569 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ad01d-d080-40dc-a158-cb4c2dcfbca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.438853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ad01d-d080-40dc-a158-cb4c2dcfbca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.439132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ad01d-d080-40dc-a158-cb4c2dcfbca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.439833 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ad01d-d080-40dc-a158-cb4c2dcfbca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:49.441419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ad01d-d080-40dc-a158-cb4c2dcfbca5 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:49.454962 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1092/2150] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:54:49.459025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ad01d-d080-40dc-a158-cb4c2dcfbca5 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:49.469812 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ad01d-d080-40dc-a158-cb4c2dcfbca5 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:49.474697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a477f-a53f-4eea-a218-0f62ae6485f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:49.475306 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a477f-a53f-4eea-a218-0f62ae6485f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.475587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a477f-a53f-4eea-a218-0f62ae6485f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.475860 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a477f-a53f-4eea-a218-0f62ae6485f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.476716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a477f-a53f-4eea-a218-0f62ae6485f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:49.478174 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a477f-a53f-4eea-a218-0f62ae6485f2 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:49.489744 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1093/2151] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:54:49.493823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a477f-a53f-4eea-a218-0f62ae6485f2 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:49.505703 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a477f-a53f-4eea-a218-0f62ae6485f2 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:49.506348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0766b-e080-4417-9426-ba634ef0e892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:49.508047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0766b-e080-4417-9426-ba634ef0e892 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:49.510221 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc53a1-61f1-4fb9-b68e-e7b1af3c36d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:49.510925 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc53a1-61f1-4fb9-b68e-e7b1af3c36d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.511195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc53a1-61f1-4fb9-b68e-e7b1af3c36d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.511459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc53a1-61f1-4fb9-b68e-e7b1af3c36d5 admin admin] PATH_INFO: `/v3/projects/9c17656e9e664e898288ec163f11fc63/users/8ccd455b417549e3b00439cee4577d11/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.521290 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1059/2152] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:54:49.526955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc53a1-61f1-4fb9-b68e-e7b1af3c36d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c17656e9e664e898288ec163f11fc63, user_id=8ccd455b417549e3b00439cee4577d11, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:49.529586 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc53a1-61f1-4fb9-b68e-e7b1af3c36d5 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:49.544882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1094/2153] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:49 2022] PUT /identity/v3/projects/9c17656e9e664e898288ec163f11fc63/users/8ccd455b417549e3b00439cee4577d11/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:49.552615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d547284-330c-49f7-a9d1-be0b8b110e6b None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.552902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d547284-330c-49f7-a9d1-be0b8b110e6b None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.554838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d547284-330c-49f7-a9d1-be0b8b110e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.576128 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-0d547284-330c-49f7-a9d1-be0b8b110e6b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:49.577419 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-0d547284-330c-49f7-a9d1-be0b8b110e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:49.593730 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc53a1-61f1-4fb9-b68e-e7b1af3c36d5 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:49.640038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1060/2154] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:49.646894 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:49.650179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d547284-330c-49f7-a9d1-be0b8b110e6b None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:49.651612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-0d547284-330c-49f7-a9d1-be0b8b110e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:49.659109 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc53a1-61f1-4fb9-b68e-e7b1af3c36d5 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:49.664516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337b131-0f3a-4c63-bb75-939113277806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:49.665180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337b131-0f3a-4c63-bb75-939113277806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.665461 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337b131-0f3a-4c63-bb75-939113277806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.665735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f337b131-0f3a-4c63-bb75-939113277806 admin admin] PATH_INFO: `/v3/users/29d2c32e972d49e0825c035771edf469` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.666925 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337b131-0f3a-4c63-bb75-939113277806 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29d2c32e972d49e0825c035771edf469)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:49.668838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f337b131-0f3a-4c63-bb75-939113277806 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:49.713321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d547284-330c-49f7-a9d1-be0b8b110e6b None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:49.716998 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76995573-fe5a-4804-83f3-9accbbcc60b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:49.717611 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76995573-fe5a-4804-83f3-9accbbcc60b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.717887 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76995573-fe5a-4804-83f3-9accbbcc60b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.718156 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76995573-fe5a-4804-83f3-9accbbcc60b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.719594 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f337b131-0f3a-4c63-bb75-939113277806 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29d2c32e972d49e0825c035771edf469'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:49.727195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f337b131-0f3a-4c63-bb75-939113277806 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29d2c32e972d49e0825c035771edf469'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:49.727811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76995573-fe5a-4804-83f3-9accbbcc60b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:49.729236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76995573-fe5a-4804-83f3-9accbbcc60b9 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:49.742263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f337b131-0f3a-4c63-bb75-939113277806 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29d2c32e972d49e0825c035771edf469'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:49.743095 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f337b131-0f3a-4c63-bb75-939113277806 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29d2c32e972d49e0825c035771edf469'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:49.743768 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f337b131-0f3a-4c63-bb75-939113277806 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29d2c32e972d49e0825c035771edf469'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:49.750729 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1061/2155] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:49.760545 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1095/2156] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:49 2022] DELETE /identity/v3/users/29d2c32e972d49e0825c035771edf469 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:49.765065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615fb78-5378-4c38-88a6-48775d0aeac1 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.765346 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615fb78-5378-4c38-88a6-48775d0aeac1 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.765618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615fb78-5378-4c38-88a6-48775d0aeac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.786362 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-6615fb78-5378-4c38-88a6-48775d0aeac1 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:49.787591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6615fb78-5378-4c38-88a6-48775d0aeac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:49.848634 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:49.871117 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1062/2157] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:54:49.881087 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337b131-0f3a-4c63-bb75-939113277806 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:49.882612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f337b131-0f3a-4c63-bb75-939113277806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:49.964404 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f337b131-0f3a-4c63-bb75-939113277806 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:49.968216 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ab5af-6074-4c8b-895b-120e03cc9b65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:49.968915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ab5af-6074-4c8b-895b-120e03cc9b65 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:49.969272 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ab5af-6074-4c8b-895b-120e03cc9b65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:49.969620 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ab5af-6074-4c8b-895b-120e03cc9b65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:49.979965 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ab5af-6074-4c8b-895b-120e03cc9b65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:49.981501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002ab5af-6074-4c8b-895b-120e03cc9b65 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:49.994097 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1096/2158] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:50.460160 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615fb78-5378-4c38-88a6-48775d0aeac1 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:50.461710 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6615fb78-5378-4c38-88a6-48775d0aeac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:50.480321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615fb78-5378-4c38-88a6-48775d0aeac1 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:50.485552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cd869-9210-4e4e-b7ed-a7847267c9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:50.486193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cd869-9210-4e4e-b7ed-a7847267c9af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:50.486474 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cd869-9210-4e4e-b7ed-a7847267c9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:50.486791 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cd869-9210-4e4e-b7ed-a7847267c9af admin admin] PATH_INFO: `/v3/projects/da55b2e081da4144aca117dbcd03d846` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:50.487955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cd869-9210-4e4e-b7ed-a7847267c9af admin admin] RBAC: Authorizing `identity:delete_project(project_id=da55b2e081da4144aca117dbcd03d846)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:50.491180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cd869-9210-4e4e-b7ed-a7847267c9af admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:50.540256 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f65cd869-9210-4e4e-b7ed-a7847267c9af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da55b2e081da4144aca117dbcd03d846', 'request_id': 'req-f65cd869-9210-4e4e-b7ed-a7847267c9af'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:50.549472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f65cd869-9210-4e4e-b7ed-a7847267c9af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da55b2e081da4144aca117dbcd03d846', 'request_id': 'req-f65cd869-9210-4e4e-b7ed-a7847267c9af'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:50.557787 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f65cd869-9210-4e4e-b7ed-a7847267c9af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da55b2e081da4144aca117dbcd03d846', 'request_id': 'req-f65cd869-9210-4e4e-b7ed-a7847267c9af'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:50.570663 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f65cd869-9210-4e4e-b7ed-a7847267c9af admin admin] The token cache is being invalidate because project da55b2e081da4144aca117dbcd03d846 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:50.570836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f65cd869-9210-4e4e-b7ed-a7847267c9af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:50.571472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f65cd869-9210-4e4e-b7ed-a7847267c9af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:50.573360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1063/2159] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:50 2022] DELETE /identity/v3/projects/da55b2e081da4144aca117dbcd03d846 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:50.577368 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ab5af-6074-4c8b-895b-120e03cc9b65 service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:50.587515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ab5af-6074-4c8b-895b-120e03cc9b65 service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:50.656722 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969645b7-b679-49bb-84b4-3deded44629e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:50.657386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969645b7-b679-49bb-84b4-3deded44629e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:50.657718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969645b7-b679-49bb-84b4-3deded44629e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:50.657998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969645b7-b679-49bb-84b4-3deded44629e admin admin] PATH_INFO: `/v3/users/98b24dfaf3454f15b9366f5c7a4c7dbc` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:50.659221 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969645b7-b679-49bb-84b4-3deded44629e admin admin] RBAC: Authorizing `identity:delete_user(user_id=98b24dfaf3454f15b9366f5c7a4c7dbc)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:50.661761 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969645b7-b679-49bb-84b4-3deded44629e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:50.675639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:50.724578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-969645b7-b679-49bb-84b4-3deded44629e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98b24dfaf3454f15b9366f5c7a4c7dbc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:50.734948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-969645b7-b679-49bb-84b4-3deded44629e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98b24dfaf3454f15b9366f5c7a4c7dbc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:50.748276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-969645b7-b679-49bb-84b4-3deded44629e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98b24dfaf3454f15b9366f5c7a4c7dbc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:50.749012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-969645b7-b679-49bb-84b4-3deded44629e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98b24dfaf3454f15b9366f5c7a4c7dbc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:50.749888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-969645b7-b679-49bb-84b4-3deded44629e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98b24dfaf3454f15b9366f5c7a4c7dbc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:50.764704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1097/2160] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:50 2022] DELETE /identity/v3/users/98b24dfaf3454f15b9366f5c7a4c7dbc => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:51.247093 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cd869-9210-4e4e-b7ed-a7847267c9af admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:51.248560 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f65cd869-9210-4e4e-b7ed-a7847267c9af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:51.291882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cd869-9210-4e4e-b7ed-a7847267c9af admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:51.376717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2630710-7b2e-41c4-ab94-9bcae1218718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:51.377425 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2630710-7b2e-41c4-ab94-9bcae1218718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:51.377732 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2630710-7b2e-41c4-ab94-9bcae1218718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:51.378008 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2630710-7b2e-41c4-ab94-9bcae1218718 admin admin] PATH_INFO: `/v3/projects/f541cf4b4a68421a81187d858e3173f8` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:51.379985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2630710-7b2e-41c4-ab94-9bcae1218718 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f541cf4b4a68421a81187d858e3173f8)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:51.382201 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2630710-7b2e-41c4-ab94-9bcae1218718 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:51.466992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c2630710-7b2e-41c4-ab94-9bcae1218718 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f541cf4b4a68421a81187d858e3173f8', 'request_id': 'req-c2630710-7b2e-41c4-ab94-9bcae1218718'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:51.482271 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c2630710-7b2e-41c4-ab94-9bcae1218718 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f541cf4b4a68421a81187d858e3173f8', 'request_id': 'req-c2630710-7b2e-41c4-ab94-9bcae1218718'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:51.494707 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c2630710-7b2e-41c4-ab94-9bcae1218718 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f541cf4b4a68421a81187d858e3173f8', 'request_id': 'req-c2630710-7b2e-41c4-ab94-9bcae1218718'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:51.512244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c2630710-7b2e-41c4-ab94-9bcae1218718 admin admin] The token cache is being invalidate because project f541cf4b4a68421a81187d858e3173f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:51.517022 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c2630710-7b2e-41c4-ab94-9bcae1218718 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:51.524497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c2630710-7b2e-41c4-ab94-9bcae1218718 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:51.527134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1064/2161] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:51 2022] DELETE /identity/v3/projects/f541cf4b4a68421a81187d858e3173f8 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:51.538691 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0146b52-659e-4796-8207-6cace106add6 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:51.539209 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0146b52-659e-4796-8207-6cace106add6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:51.539554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0146b52-659e-4796-8207-6cace106add6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:51.576063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-d0146b52-659e-4796-8207-6cace106add6 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:51.584698 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d0146b52-659e-4796-8207-6cace106add6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:51.686283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1098/2162] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:51.688594 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc500b-6808-4c02-885a-2453d8ea75ea None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:51.690712 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc500b-6808-4c02-885a-2453d8ea75ea None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:51.696077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc500b-6808-4c02-885a-2453d8ea75ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:51.717203 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:51.732488 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-adfc500b-6808-4c02-885a-2453d8ea75ea None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:51.733728 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-adfc500b-6808-4c02-885a-2453d8ea75ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:51.745762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1065/2163] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:51.748339 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0146b52-659e-4796-8207-6cace106add6 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:51.760003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0146b52-659e-4796-8207-6cace106add6 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:51.765450 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e2109-ac2b-4a28-a253-bf11312415f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:51.766419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e2109-ac2b-4a28-a253-bf11312415f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:51.766768 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e2109-ac2b-4a28-a253-bf11312415f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:51.767051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e2109-ac2b-4a28-a253-bf11312415f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:51.767764 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e2109-ac2b-4a28-a253-bf11312415f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:54:51.768071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e2109-ac2b-4a28-a253-bf11312415f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:51.770190 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e2109-ac2b-4a28-a253-bf11312415f7 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:51.783195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1099/2164] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:54:51.787446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc500b-6808-4c02-885a-2453d8ea75ea None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:51.796846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc500b-6808-4c02-885a-2453d8ea75ea None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:51.802911 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd15f3e-0201-4091-a29e-f315cae2bd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:51.803604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd15f3e-0201-4091-a29e-f315cae2bd36 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:51.803955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd15f3e-0201-4091-a29e-f315cae2bd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:51.804298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd15f3e-0201-4091-a29e-f315cae2bd36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:51.804992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd15f3e-0201-4091-a29e-f315cae2bd36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:51.806919 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd15f3e-0201-4091-a29e-f315cae2bd36 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:51.834607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1066/2165] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:51 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:51.838416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e2109-ac2b-4a28-a253-bf11312415f7 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:51.849184 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e2109-ac2b-4a28-a253-bf11312415f7 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:51.923977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb098e46-c70f-42b3-86fa-3344a7dd349e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:51.924611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb098e46-c70f-42b3-86fa-3344a7dd349e admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:51.924892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb098e46-c70f-42b3-86fa-3344a7dd349e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:51.925182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb098e46-c70f-42b3-86fa-3344a7dd349e admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:51.926705 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb098e46-c70f-42b3-86fa-3344a7dd349e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:51.928602 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb098e46-c70f-42b3-86fa-3344a7dd349e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:51.949619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1100/2166] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:51 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:51.953944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd15f3e-0201-4091-a29e-f315cae2bd36 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:51.967053 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd15f3e-0201-4091-a29e-f315cae2bd36 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:51.971808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b61e8f-3fd9-4deb-adf1-9f1482e6e584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:51.972432 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b61e8f-3fd9-4deb-adf1-9f1482e6e584 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:51.973229 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b61e8f-3fd9-4deb-adf1-9f1482e6e584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:51.973505 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b61e8f-3fd9-4deb-adf1-9f1482e6e584 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:51.974214 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b61e8f-3fd9-4deb-adf1-9f1482e6e584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:51.975846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b61e8f-3fd9-4deb-adf1-9f1482e6e584 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:51.986475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1067/2167] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:51.991294 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb098e46-c70f-42b3-86fa-3344a7dd349e admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:52.001233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb098e46-c70f-42b3-86fa-3344a7dd349e admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:52.005856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd5e310-51c0-41b3-bbb7-2a1a8bdef8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:52.006475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5e310-51c0-41b3-bbb7-2a1a8bdef8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:52.006791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5e310-51c0-41b3-bbb7-2a1a8bdef8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:52.007079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd5e310-51c0-41b3-bbb7-2a1a8bdef8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:52.007763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5e310-51c0-41b3-bbb7-2a1a8bdef8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:52.009374 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd5e310-51c0-41b3-bbb7-2a1a8bdef8e7 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:52.010771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:52.024606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1101/2168] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:52.025727 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b61e8f-3fd9-4deb-adf1-9f1482e6e584 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:52.038533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b61e8f-3fd9-4deb-adf1-9f1482e6e584 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:52.043642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f8381-36d2-439d-9dff-677bddd83414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:52.044326 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f8381-36d2-439d-9dff-677bddd83414 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:52.044604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f8381-36d2-439d-9dff-677bddd83414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:52.044879 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049f8381-36d2-439d-9dff-677bddd83414 admin admin] PATH_INFO: `/v3/projects/7108b93261984747a0dbaa4251701937/users/66a9cba05b0d480ba6475600b9ba1d69/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:52.065005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f8381-36d2-439d-9dff-677bddd83414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7108b93261984747a0dbaa4251701937, user_id=66a9cba05b0d480ba6475600b9ba1d69, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:52.067986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049f8381-36d2-439d-9dff-677bddd83414 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:52.082539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1068/2169] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:52 2022] PUT /identity/v3/projects/7108b93261984747a0dbaa4251701937/users/66a9cba05b0d480ba6475600b9ba1d69/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:52.094152 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf344e-3608-49b4-a46e-bcae663584e5 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:52.094385 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf344e-3608-49b4-a46e-bcae663584e5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:52.094601 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf344e-3608-49b4-a46e-bcae663584e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:52.129781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-a5bf344e-3608-49b4-a46e-bcae663584e5 None None] MFA Rules not processed for user `66a9cba05b0d480ba6475600b9ba1d69`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:52.131046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a5bf344e-3608-49b4-a46e-bcae663584e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:52.216815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1102/2170] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:52.221142 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f8381-36d2-439d-9dff-677bddd83414 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:52.295948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049f8381-36d2-439d-9dff-677bddd83414 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:52.306269 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18cb25-0043-4bda-8758-05a63a821f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:52.306959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18cb25-0043-4bda-8758-05a63a821f96 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:52.307240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18cb25-0043-4bda-8758-05a63a821f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:52.307515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18cb25-0043-4bda-8758-05a63a821f96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:52.308206 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18cb25-0043-4bda-8758-05a63a821f96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:52.310145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18cb25-0043-4bda-8758-05a63a821f96 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:52.348805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1069/2171] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:52 2022] POST /identity/v3/projects => generated 389 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:54:52.359369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf344e-3608-49b4-a46e-bcae663584e5 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:52.372213 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf344e-3608-49b4-a46e-bcae663584e5 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:52.443970 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c844a69-e63b-49d6-8b5b-fad5f07ed33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:52.444676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c844a69-e63b-49d6-8b5b-fad5f07ed33b admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:52.444970 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c844a69-e63b-49d6-8b5b-fad5f07ed33b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:52.445242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c844a69-e63b-49d6-8b5b-fad5f07ed33b admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:52.445922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c844a69-e63b-49d6-8b5b-fad5f07ed33b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:52.448075 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c844a69-e63b-49d6-8b5b-fad5f07ed33b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:52.479986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18cb25-0043-4bda-8758-05a63a821f96 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:52.482561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1103/2172] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:52 2022] POST /identity/v3/users => generated 301 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:52.492295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18cb25-0043-4bda-8758-05a63a821f96 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:52.497420 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3bb5f-ce0f-436c-8c59-213c38278e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:52.498078 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3bb5f-ce0f-436c-8c59-213c38278e5f admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:52.498354 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3bb5f-ce0f-436c-8c59-213c38278e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:52.498662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3bb5f-ce0f-436c-8c59-213c38278e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:52.499808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3bb5f-ce0f-436c-8c59-213c38278e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:52.501587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3bb5f-ce0f-436c-8c59-213c38278e5f admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:52.516125 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1070/2173] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:54:52.518769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c844a69-e63b-49d6-8b5b-fad5f07ed33b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:52.532664 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c844a69-e63b-49d6-8b5b-fad5f07ed33b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:52.537992 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c5ea5-2cc7-4914-8a4c-ad08a637611b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:52.538706 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c5ea5-2cc7-4914-8a4c-ad08a637611b admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:52.538987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c5ea5-2cc7-4914-8a4c-ad08a637611b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:52.539260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c5ea5-2cc7-4914-8a4c-ad08a637611b admin admin] PATH_INFO: `/v3/projects/6aa940b9d557489e8255ceccaf702ac9/users/35eeb97e680c4c03b848c7c1bdc23b10/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:52.561011 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c5ea5-2cc7-4914-8a4c-ad08a637611b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aa940b9d557489e8255ceccaf702ac9, user_id=35eeb97e680c4c03b848c7c1bdc23b10, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:52.566289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879c5ea5-2cc7-4914-8a4c-ad08a637611b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:52.599348 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1104/2174] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:52 2022] PUT /identity/v3/projects/6aa940b9d557489e8255ceccaf702ac9/users/35eeb97e680c4c03b848c7c1bdc23b10/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:52.601286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3bb5f-ce0f-436c-8c59-213c38278e5f admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:52.614510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3bb5f-ce0f-436c-8c59-213c38278e5f admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:52.765063 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9407813-6b3f-40c7-a273-0f9925e805ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:52.767086 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9407813-6b3f-40c7-a273-0f9925e805ff admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:52.767527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9407813-6b3f-40c7-a273-0f9925e805ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:52.777444 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9407813-6b3f-40c7-a273-0f9925e805ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:52.778303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9407813-6b3f-40c7-a273-0f9925e805ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:52.787038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9407813-6b3f-40c7-a273-0f9925e805ff admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:52.822511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1071/2175] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:52.839947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c5ea5-2cc7-4914-8a4c-ad08a637611b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:52.851126 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c5ea5-2cc7-4914-8a4c-ad08a637611b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:52.856857 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379511a-1f43-4224-9d23-d248e5eb5e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:52.857758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379511a-1f43-4224-9d23-d248e5eb5e39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:52.858221 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379511a-1f43-4224-9d23-d248e5eb5e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:52.858845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379511a-1f43-4224-9d23-d248e5eb5e39 admin admin] PATH_INFO: `/v3/projects/6aa940b9d557489e8255ceccaf702ac9/users/35eeb97e680c4c03b848c7c1bdc23b10/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:52.861281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379511a-1f43-4224-9d23-d248e5eb5e39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aa940b9d557489e8255ceccaf702ac9, user_id=35eeb97e680c4c03b848c7c1bdc23b10, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:52.864525 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379511a-1f43-4224-9d23-d248e5eb5e39 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:52.883658 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1105/2176] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:52 2022] PUT /identity/v3/projects/6aa940b9d557489e8255ceccaf702ac9/users/35eeb97e680c4c03b848c7c1bdc23b10/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:52.895635 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2a4f8-6650-4131-80e8-5a005bca9646 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:52.896140 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2a4f8-6650-4131-80e8-5a005bca9646 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:52.896492 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a2a4f8-6650-4131-80e8-5a005bca9646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:52.931423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:52.948924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a1a2a4f8-6650-4131-80e8-5a005bca9646 None None] MFA Rules not processed for user `35eeb97e680c4c03b848c7c1bdc23b10`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:52.950997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a1a2a4f8-6650-4131-80e8-5a005bca9646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:53.087277 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1072/2177] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:52 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:53.103953 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379511a-1f43-4224-9d23-d248e5eb5e39 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:53.107620 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e379511a-1f43-4224-9d23-d248e5eb5e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:53.209617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:53.228552 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379511a-1f43-4224-9d23-d248e5eb5e39 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:53.233181 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e28ff-21c5-494b-b8ce-a3cae9b296a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:53.234201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e28ff-21c5-494b-b8ce-a3cae9b296a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:53.235408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e28ff-21c5-494b-b8ce-a3cae9b296a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:53.236020 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e28ff-21c5-494b-b8ce-a3cae9b296a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:53.258418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e28ff-21c5-494b-b8ce-a3cae9b296a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:53.262383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e28ff-21c5-494b-b8ce-a3cae9b296a7 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:53.291828 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1106/2178] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:53 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:53.491243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2a4f8-6650-4131-80e8-5a005bca9646 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:53.573676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a2a4f8-6650-4131-80e8-5a005bca9646 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:53.580112 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4bbcd-6cf2-4a70-8bf8-f97554af7567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:53.580768 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4bbcd-6cf2-4a70-8bf8-f97554af7567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:53.581047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4bbcd-6cf2-4a70-8bf8-f97554af7567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:53.581653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4bbcd-6cf2-4a70-8bf8-f97554af7567 admin admin] PATH_INFO: `/v3/users/66a9cba05b0d480ba6475600b9ba1d69` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:53.582905 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4bbcd-6cf2-4a70-8bf8-f97554af7567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66a9cba05b0d480ba6475600b9ba1d69)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:53.584938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4bbcd-6cf2-4a70-8bf8-f97554af7567 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:53.657980 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-6ca4bbcd-6cf2-4a70-8bf8-f97554af7567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66a9cba05b0d480ba6475600b9ba1d69'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:53.679880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-6ca4bbcd-6cf2-4a70-8bf8-f97554af7567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66a9cba05b0d480ba6475600b9ba1d69'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:53.697753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-6ca4bbcd-6cf2-4a70-8bf8-f97554af7567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66a9cba05b0d480ba6475600b9ba1d69'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:53.700652 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-6ca4bbcd-6cf2-4a70-8bf8-f97554af7567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66a9cba05b0d480ba6475600b9ba1d69'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:53.729033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-6ca4bbcd-6cf2-4a70-8bf8-f97554af7567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66a9cba05b0d480ba6475600b9ba1d69'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:53.747679 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252a3ed-8414-4b88-b6c3-8dbf3ddd6d34 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:53.748130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252a3ed-8414-4b88-b6c3-8dbf3ddd6d34 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:53.748233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5252a3ed-8414-4b88-b6c3-8dbf3ddd6d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:53.751617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1073/2179] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:53 2022] DELETE /identity/v3/users/66a9cba05b0d480ba6475600b9ba1d69 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:53.774724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-5252a3ed-8414-4b88-b6c3-8dbf3ddd6d34 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:53.775887 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5252a3ed-8414-4b88-b6c3-8dbf3ddd6d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:53.845809 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1107/2180] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:53.858635 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4bbcd-6cf2-4a70-8bf8-f97554af7567 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:53.860647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6ca4bbcd-6cf2-4a70-8bf8-f97554af7567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:53.895740 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:53.979143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4bbcd-6cf2-4a70-8bf8-f97554af7567 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:53.985718 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d512ef-6c34-459b-b488-60008ebbaef3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:53.987077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d512ef-6c34-459b-b488-60008ebbaef3 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:53.987403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d512ef-6c34-459b-b488-60008ebbaef3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:53.987771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d512ef-6c34-459b-b488-60008ebbaef3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:54.005127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d512ef-6c34-459b-b488-60008ebbaef3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:54.006605 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d512ef-6c34-459b-b488-60008ebbaef3 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:54.020936 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1074/2181] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:54.615812 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252a3ed-8414-4b88-b6c3-8dbf3ddd6d34 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:54.617396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5252a3ed-8414-4b88-b6c3-8dbf3ddd6d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:54.631518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5252a3ed-8414-4b88-b6c3-8dbf3ddd6d34 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:54.638239 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:54.639468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:54.639468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:54.639758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd admin admin] PATH_INFO: `/v3/projects/7108b93261984747a0dbaa4251701937` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:54.641998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7108b93261984747a0dbaa4251701937)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:54.645366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:54.709768 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7108b93261984747a0dbaa4251701937', 'request_id': 'req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:54.719138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7108b93261984747a0dbaa4251701937', 'request_id': 'req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:54.725982 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7108b93261984747a0dbaa4251701937', 'request_id': 'req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:54.737904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd admin admin] The token cache is being invalidate because project 7108b93261984747a0dbaa4251701937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:54.739011 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:54.739759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:54.744760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1108/2182] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:54 2022] DELETE /identity/v3/projects/7108b93261984747a0dbaa4251701937 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:54.745993 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d512ef-6c34-459b-b488-60008ebbaef3 service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:54.780696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d512ef-6c34-459b-b488-60008ebbaef3 service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:54.883117 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3a8c6-a45c-45b7-9948-c0a3d777b423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:54.883785 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3a8c6-a45c-45b7-9948-c0a3d777b423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:54.884063 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3a8c6-a45c-45b7-9948-c0a3d777b423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:54.884336 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3a8c6-a45c-45b7-9948-c0a3d777b423 admin admin] PATH_INFO: `/v3/users/35eeb97e680c4c03b848c7c1bdc23b10` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:54.885603 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb3a8c6-a45c-45b7-9948-c0a3d777b423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35eeb97e680c4c03b848c7c1bdc23b10)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:54.887681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb3a8c6-a45c-45b7-9948-c0a3d777b423 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:54.962216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-aeb3a8c6-a45c-45b7-9948-c0a3d777b423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35eeb97e680c4c03b848c7c1bdc23b10'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:54.984904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-aeb3a8c6-a45c-45b7-9948-c0a3d777b423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35eeb97e680c4c03b848c7c1bdc23b10'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:54.996801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-aeb3a8c6-a45c-45b7-9948-c0a3d777b423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35eeb97e680c4c03b848c7c1bdc23b10'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:54.997318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:54.998468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-aeb3a8c6-a45c-45b7-9948-c0a3d777b423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35eeb97e680c4c03b848c7c1bdc23b10'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:54.999644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:55.030137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d58e4-9577-4ee4-9b08-61f3b97c14bd admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:55.030260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-aeb3a8c6-a45c-45b7-9948-c0a3d777b423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35eeb97e680c4c03b848c7c1bdc23b10'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:55.037856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ccb93-e401-45da-b372-4a6478fac965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:55.039180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ccb93-e401-45da-b372-4a6478fac965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:55.040009 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ccb93-e401-45da-b372-4a6478fac965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:55.040359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ccb93-e401-45da-b372-4a6478fac965 admin admin] PATH_INFO: `/v3/users/49548220763c47b58d5ae6870e5e042f` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:55.040974 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1075/2183] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:54 2022] DELETE /identity/v3/users/35eeb97e680c4c03b848c7c1bdc23b10 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:55.046355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ccb93-e401-45da-b372-4a6478fac965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49548220763c47b58d5ae6870e5e042f)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:55.049788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ccb93-e401-45da-b372-4a6478fac965 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:55.182747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-538ccb93-e401-45da-b372-4a6478fac965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49548220763c47b58d5ae6870e5e042f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:55.194707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-538ccb93-e401-45da-b372-4a6478fac965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49548220763c47b58d5ae6870e5e042f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:55.200083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:55.230652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-538ccb93-e401-45da-b372-4a6478fac965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49548220763c47b58d5ae6870e5e042f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:55.233387 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-538ccb93-e401-45da-b372-4a6478fac965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49548220763c47b58d5ae6870e5e042f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:55.241152 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-538ccb93-e401-45da-b372-4a6478fac965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49548220763c47b58d5ae6870e5e042f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:55.272707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1109/2184] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:54 2022] DELETE /identity/v3/users/49548220763c47b58d5ae6870e5e042f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:55.276429 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0551d3b-8588-4208-8d65-0d65c2c7867c None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:55.276713 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0551d3b-8588-4208-8d65-0d65c2c7867c None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:55.277014 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0551d3b-8588-4208-8d65-0d65c2c7867c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:55.320874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-b0551d3b-8588-4208-8d65-0d65c2c7867c None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:55.327254 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-b0551d3b-8588-4208-8d65-0d65c2c7867c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:55.412871 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:55.483764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1076/2185] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:55.498742 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ccb93-e401-45da-b372-4a6478fac965 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:55.500460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-538ccb93-e401-45da-b372-4a6478fac965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:55.529457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c46fa-2305-4397-9588-365696e44d78 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:55.529771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c46fa-2305-4397-9588-365696e44d78 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:55.530011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c46fa-2305-4397-9588-365696e44d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:55.600716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-8d6c46fa-2305-4397-9588-365696e44d78 None None] MFA Rules not processed for user `8ccd455b417549e3b00439cee4577d11`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:55.602624 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8d6c46fa-2305-4397-9588-365696e44d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:55.683255 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ccb93-e401-45da-b372-4a6478fac965 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:55.695274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48e4ee-20da-4731-af44-3417015a4f43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:55.695808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48e4ee-20da-4731-af44-3417015a4f43 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:55.696095 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48e4ee-20da-4731-af44-3417015a4f43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:55.696372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48e4ee-20da-4731-af44-3417015a4f43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:55.710127 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48e4ee-20da-4731-af44-3417015a4f43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:55.711403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48e4ee-20da-4731-af44-3417015a4f43 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:55.724426 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1077/2186] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:54:55.725006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1110/2187] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:55.725828 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48e4ee-20da-4731-af44-3417015a4f43 service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:55.725967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c46fa-2305-4397-9588-365696e44d78 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:55.727755 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1f48e4ee-20da-4731-af44-3417015a4f43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:55.728791 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8d6c46fa-2305-4397-9588-365696e44d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:55.742469 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48e4ee-20da-4731-af44-3417015a4f43 service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:55.747295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c46fa-2305-4397-9588-365696e44d78 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:55.749139 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ca98c-89e5-486b-8e5e-a44697088b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:55.749769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ca98c-89e5-486b-8e5e-a44697088b7e admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:55.750099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ca98c-89e5-486b-8e5e-a44697088b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:55.750317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ca98c-89e5-486b-8e5e-a44697088b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:55.751175 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ca98c-89e5-486b-8e5e-a44697088b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:55.753711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ca98c-89e5-486b-8e5e-a44697088b7e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:55.758145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce902e-da29-44f0-8da0-19a3fd29091c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:55.768597 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce902e-da29-44f0-8da0-19a3fd29091c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:55.774967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce902e-da29-44f0-8da0-19a3fd29091c admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:55.775563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce902e-da29-44f0-8da0-19a3fd29091c admin admin] PATH_INFO: `/v3/projects/6aa940b9d557489e8255ceccaf702ac9` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:55.778284 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce902e-da29-44f0-8da0-19a3fd29091c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aa940b9d557489e8255ceccaf702ac9)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:55.782304 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce902e-da29-44f0-8da0-19a3fd29091c admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:55.813638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1111/2188] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:55 2022] POST /identity/v3/projects => generated 377 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:55.819050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ca98c-89e5-486b-8e5e-a44697088b7e admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:55.905603 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cdce902e-da29-44f0-8da0-19a3fd29091c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aa940b9d557489e8255ceccaf702ac9', 'request_id': 'req-cdce902e-da29-44f0-8da0-19a3fd29091c'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:55.922737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cdce902e-da29-44f0-8da0-19a3fd29091c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aa940b9d557489e8255ceccaf702ac9', 'request_id': 'req-cdce902e-da29-44f0-8da0-19a3fd29091c'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:55.932517 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cdce902e-da29-44f0-8da0-19a3fd29091c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aa940b9d557489e8255ceccaf702ac9', 'request_id': 'req-cdce902e-da29-44f0-8da0-19a3fd29091c'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:55.938357 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ca98c-89e5-486b-8e5e-a44697088b7e admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:55.960580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cdce902e-da29-44f0-8da0-19a3fd29091c admin admin] The token cache is being invalidate because project 6aa940b9d557489e8255ceccaf702ac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:55.960973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cdce902e-da29-44f0-8da0-19a3fd29091c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:55.961833 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cdce902e-da29-44f0-8da0-19a3fd29091c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:55.965333 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1078/2189] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:55 2022] DELETE /identity/v3/projects/6aa940b9d557489e8255ceccaf702ac9 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:55.971670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad7da1-7c30-46ae-8d90-412351abb436 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:55.972087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad7da1-7c30-46ae-8d90-412351abb436 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:55.973821 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad7da1-7c30-46ae-8d90-412351abb436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:56.013928 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-ccad7da1-7c30-46ae-8d90-412351abb436 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:56.015705 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ccad7da1-7c30-46ae-8d90-412351abb436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:56.035659 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc24ce-7e9a-4095-8fde-df558e093d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:56.036522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc24ce-7e9a-4095-8fde-df558e093d6a admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:56.036862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc24ce-7e9a-4095-8fde-df558e093d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:56.037250 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc24ce-7e9a-4095-8fde-df558e093d6a admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:56.038046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc24ce-7e9a-4095-8fde-df558e093d6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:56.040256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc24ce-7e9a-4095-8fde-df558e093d6a admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:56.052550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:56.079997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1112/2190] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:55 2022] POST /identity/v3/users => generated 295 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:56.080816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc24ce-7e9a-4095-8fde-df558e093d6a admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:56.082903 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1ddc24ce-7e9a-4095-8fde-df558e093d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:56.103387 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc24ce-7e9a-4095-8fde-df558e093d6a admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:56.110534 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371dab09-a79b-4c04-b15f-a6f3e5ad0a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:56.111355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371dab09-a79b-4c04-b15f-a6f3e5ad0a81 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:56.111640 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371dab09-a79b-4c04-b15f-a6f3e5ad0a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:56.111911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371dab09-a79b-4c04-b15f-a6f3e5ad0a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:56.112773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371dab09-a79b-4c04-b15f-a6f3e5ad0a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:56.114786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371dab09-a79b-4c04-b15f-a6f3e5ad0a81 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:56.137207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:56.150527 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1113/2191] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:56.155252 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371dab09-a79b-4c04-b15f-a6f3e5ad0a81 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:56.171107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371dab09-a79b-4c04-b15f-a6f3e5ad0a81 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:56.177542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1079/2192] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:56.182060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7ec26-49cf-4e20-8b13-57e956b5a090 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:56.183000 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7ec26-49cf-4e20-8b13-57e956b5a090 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:56.183865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7ec26-49cf-4e20-8b13-57e956b5a090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:56.188026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cddc31-e98a-4e39-8cdf-5a22e80ce6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:56.190382 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cddc31-e98a-4e39-8cdf-5a22e80ce6eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:56.192064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cddc31-e98a-4e39-8cdf-5a22e80ce6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:56.193323 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cddc31-e98a-4e39-8cdf-5a22e80ce6eb admin admin] PATH_INFO: `/v3/projects/05a3a60014f445feb168c25b4109cc9b/users/b142cc45cfab477eb8dfb40f817c9f27/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:56.228965 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-d5a7ec26-49cf-4e20-8b13-57e956b5a090 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:56.232888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cddc31-e98a-4e39-8cdf-5a22e80ce6eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a3a60014f445feb168c25b4109cc9b, user_id=b142cc45cfab477eb8dfb40f817c9f27, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:56.234542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d5a7ec26-49cf-4e20-8b13-57e956b5a090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:56.236563 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cddc31-e98a-4e39-8cdf-5a22e80ce6eb admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:56.257499 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1080/2193] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:56.261750 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7ec26-49cf-4e20-8b13-57e956b5a090 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:56.268123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1114/2194] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:56 2022] PUT /identity/v3/projects/05a3a60014f445feb168c25b4109cc9b/users/b142cc45cfab477eb8dfb40f817c9f27/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:56.273311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cddc31-e98a-4e39-8cdf-5a22e80ce6eb admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:56.282348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7ec26-49cf-4e20-8b13-57e956b5a090 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:56.371071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cddc31-e98a-4e39-8cdf-5a22e80ce6eb admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:56.376837 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80420e42-13d0-4996-8912-07314c464598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:56.377913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80420e42-13d0-4996-8912-07314c464598 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:56.378257 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80420e42-13d0-4996-8912-07314c464598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:56.378632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80420e42-13d0-4996-8912-07314c464598 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:56.379482 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80420e42-13d0-4996-8912-07314c464598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:56.381216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8b5cd-dd3b-465d-bc4b-ff29a5bee296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:56.381354 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80420e42-13d0-4996-8912-07314c464598 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:56.381979 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8b5cd-dd3b-465d-bc4b-ff29a5bee296 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:56.382337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8b5cd-dd3b-465d-bc4b-ff29a5bee296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:56.382750 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8b5cd-dd3b-465d-bc4b-ff29a5bee296 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:56.384370 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8b5cd-dd3b-465d-bc4b-ff29a5bee296 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:54:56.384759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8b5cd-dd3b-465d-bc4b-ff29a5bee296 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:56.387759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c8b5cd-dd3b-465d-bc4b-ff29a5bee296 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:56.398960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1115/2195] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:56.398960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80420e42-13d0-4996-8912-07314c464598 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:56.398960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-80420e42-13d0-4996-8912-07314c464598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:56.413268 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1081/2196] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:54:56.414650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8b5cd-dd3b-465d-bc4b-ff29a5bee296 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:56.419775 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80420e42-13d0-4996-8912-07314c464598 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:56.427526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8b5cd-dd3b-465d-bc4b-ff29a5bee296 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:56.427677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1e1d2-ad57-4ea4-9899-e2b997114268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:56.428362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1e1d2-ad57-4ea4-9899-e2b997114268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:56.428714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1e1d2-ad57-4ea4-9899-e2b997114268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:56.428975 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1e1d2-ad57-4ea4-9899-e2b997114268 admin admin] PATH_INFO: `/v3/projects/dc4183d3440141db95f43ea9c75bb41a` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:56.430223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1e1d2-ad57-4ea4-9899-e2b997114268 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc4183d3440141db95f43ea9c75bb41a)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:56.432225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1e1d2-ad57-4ea4-9899-e2b997114268 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:56.433153 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff1744-233c-426f-884b-a59fa6921ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:56.434050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff1744-233c-426f-884b-a59fa6921ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:56.434343 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff1744-233c-426f-884b-a59fa6921ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:56.434693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff1744-233c-426f-884b-a59fa6921ac8 admin admin] PATH_INFO: `/v3/projects/05a3a60014f445feb168c25b4109cc9b/users/b142cc45cfab477eb8dfb40f817c9f27/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:56.437516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff1744-233c-426f-884b-a59fa6921ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a3a60014f445feb168c25b4109cc9b, user_id=b142cc45cfab477eb8dfb40f817c9f27, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:56.440778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff1744-233c-426f-884b-a59fa6921ac8 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:56.457848 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1082/2197] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:56 2022] PUT /identity/v3/projects/05a3a60014f445feb168c25b4109cc9b/users/b142cc45cfab477eb8dfb40f817c9f27/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:56.458773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff1744-233c-426f-884b-a59fa6921ac8 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:56.525679 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a3a1e1d2-ad57-4ea4-9899-e2b997114268 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc4183d3440141db95f43ea9c75bb41a', 'request_id': 'req-a3a1e1d2-ad57-4ea4-9899-e2b997114268'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:56.541856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a3a1e1d2-ad57-4ea4-9899-e2b997114268 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc4183d3440141db95f43ea9c75bb41a', 'request_id': 'req-a3a1e1d2-ad57-4ea4-9899-e2b997114268'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:56.554835 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a3a1e1d2-ad57-4ea4-9899-e2b997114268 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc4183d3440141db95f43ea9c75bb41a', 'request_id': 'req-a3a1e1d2-ad57-4ea4-9899-e2b997114268'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:56.574367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff1744-233c-426f-884b-a59fa6921ac8 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:56.618383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a3a1e1d2-ad57-4ea4-9899-e2b997114268 admin admin] The token cache is being invalidate because project dc4183d3440141db95f43ea9c75bb41a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:56.619099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a3a1e1d2-ad57-4ea4-9899-e2b997114268 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:56.620283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a3a1e1d2-ad57-4ea4-9899-e2b997114268 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:56.623507 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1116/2198] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:56 2022] DELETE /identity/v3/projects/dc4183d3440141db95f43ea9c75bb41a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:56.628319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572967d-77c3-4b44-a2e4-0f44d6da269e None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:56.628866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572967d-77c3-4b44-a2e4-0f44d6da269e None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:56.629403 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572967d-77c3-4b44-a2e4-0f44d6da269e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:56.657774 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-4572967d-77c3-4b44-a2e4-0f44d6da269e None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:56.658972 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febafea-5b64-4916-a3b2-a43d4c1aab8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:56.659661 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febafea-5b64-4916-a3b2-a43d4c1aab8d admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:56.659920 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-4572967d-77c3-4b44-a2e4-0f44d6da269e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:56.660562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febafea-5b64-4916-a3b2-a43d4c1aab8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:56.662168 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febafea-5b64-4916-a3b2-a43d4c1aab8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:56.662924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febafea-5b64-4916-a3b2-a43d4c1aab8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:56.664908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febafea-5b64-4916-a3b2-a43d4c1aab8d admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:56.687651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1117/2199] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:56.698247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb3602-130a-4579-9df0-426db46b0ff9 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:56.699101 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb3602-130a-4579-9df0-426db46b0ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:56.699385 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb3602-130a-4579-9df0-426db46b0ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:56.706519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:56.729457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1083/2200] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:56 2022] POST /identity/v3/projects => generated 385 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:56.733690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febafea-5b64-4916-a3b2-a43d4c1aab8d admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:56.739406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-0febafea-5b64-4916-a3b2-a43d4c1aab8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:56.745096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-45bb3602-130a-4579-9df0-426db46b0ff9 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:56.746460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-45bb3602-130a-4579-9df0-426db46b0ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:56.792560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:56.842323 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febafea-5b64-4916-a3b2-a43d4c1aab8d admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:56.852247 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66892e0-1223-46df-99ac-4f70f9116edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:56.852904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66892e0-1223-46df-99ac-4f70f9116edf admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:56.854327 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66892e0-1223-46df-99ac-4f70f9116edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:56.854636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66892e0-1223-46df-99ac-4f70f9116edf admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:56.855339 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66892e0-1223-46df-99ac-4f70f9116edf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:56.859059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66892e0-1223-46df-99ac-4f70f9116edf admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:56.886847 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1118/2201] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:56.889696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb3602-130a-4579-9df0-426db46b0ff9 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:56.896713 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1084/2202] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:56 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:54:56.898837 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66892e0-1223-46df-99ac-4f70f9116edf admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:56.904938 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb3602-130a-4579-9df0-426db46b0ff9 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:56.918673 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66892e0-1223-46df-99ac-4f70f9116edf admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:56.920807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959257b-0ab3-44d6-8dc1-7d74b968dd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:56.921442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959257b-0ab3-44d6-8dc1-7d74b968dd4a admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:56.921716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959257b-0ab3-44d6-8dc1-7d74b968dd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:56.921983 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959257b-0ab3-44d6-8dc1-7d74b968dd4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:56.922714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959257b-0ab3-44d6-8dc1-7d74b968dd4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:54:56.923012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959257b-0ab3-44d6-8dc1-7d74b968dd4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:56.924608 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959257b-0ab3-44d6-8dc1-7d74b968dd4a admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:56.929769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e517b1-4291-41f5-b3cb-7b776b899078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:56.939867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1119/2203] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:54:56.940926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e517b1-4291-41f5-b3cb-7b776b899078 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:56.941375 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e517b1-4291-41f5-b3cb-7b776b899078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:56.942398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e517b1-4291-41f5-b3cb-7b776b899078 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:56.943240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e517b1-4291-41f5-b3cb-7b776b899078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:56.946023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e517b1-4291-41f5-b3cb-7b776b899078 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:56.952519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959257b-0ab3-44d6-8dc1-7d74b968dd4a admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:56.960820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1085/2204] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:56.969476 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959257b-0ab3-44d6-8dc1-7d74b968dd4a admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:56.976083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bfaeb-acd1-4311-b745-cff7d84856a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:56.976913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bfaeb-acd1-4311-b745-cff7d84856a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:56.977279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bfaeb-acd1-4311-b745-cff7d84856a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:56.977637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1bfaeb-acd1-4311-b745-cff7d84856a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:56.978389 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1bfaeb-acd1-4311-b745-cff7d84856a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:56.980903 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1bfaeb-acd1-4311-b745-cff7d84856a3 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:56.991491 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e517b1-4291-41f5-b3cb-7b776b899078 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:57.007854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e517b1-4291-41f5-b3cb-7b776b899078 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:57.016149 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849227a-1faf-40d5-8c33-62537c796088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:57.017644 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849227a-1faf-40d5-8c33-62537c796088 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:57.017762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849227a-1faf-40d5-8c33-62537c796088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:57.017948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849227a-1faf-40d5-8c33-62537c796088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:57.018849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849227a-1faf-40d5-8c33-62537c796088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:57.021294 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849227a-1faf-40d5-8c33-62537c796088 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:57.026969 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1120/2205] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:56 2022] POST /identity/v3/projects => generated 359 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:57.030731 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bfaeb-acd1-4311-b745-cff7d84856a3 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:57.033226 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1086/2206] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:57.039613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849227a-1faf-40d5-8c33-62537c796088 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:57.069608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849227a-1faf-40d5-8c33-62537c796088 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:57.123867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1bfaeb-acd1-4311-b745-cff7d84856a3 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:57.130149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b110a1-814f-48aa-9cd4-785d21cfa130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:57.130912 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b110a1-814f-48aa-9cd4-785d21cfa130 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:57.131453 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b110a1-814f-48aa-9cd4-785d21cfa130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:57.131806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b110a1-814f-48aa-9cd4-785d21cfa130 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:57.132554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b110a1-814f-48aa-9cd4-785d21cfa130 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:57.135246 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b110a1-814f-48aa-9cd4-785d21cfa130 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:57.155149 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b3438-0c71-4fff-b49e-9bef25dd809e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:57.155403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b3438-0c71-4fff-b49e-9bef25dd809e admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:57.155907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b3438-0c71-4fff-b49e-9bef25dd809e admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:57.156200 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b3438-0c71-4fff-b49e-9bef25dd809e admin admin] PATH_INFO: `/v3/projects/f8659387819c43868a0718ff342ebda5/users/b4691caf4e5c4bc299885a1380bd087a/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:57.189222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b3438-0c71-4fff-b49e-9bef25dd809e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8659387819c43868a0718ff342ebda5, user_id=b4691caf4e5c4bc299885a1380bd087a, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:57.191430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b3438-0c71-4fff-b49e-9bef25dd809e admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:57.191993 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1121/2207] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:57 2022] POST /identity/v3/users => generated 280 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:54:57.193246 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b110a1-814f-48aa-9cd4-785d21cfa130 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:57.207507 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b110a1-814f-48aa-9cd4-785d21cfa130 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:57.222438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1087/2208] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:57 2022] PUT /identity/v3/projects/f8659387819c43868a0718ff342ebda5/users/b4691caf4e5c4bc299885a1380bd087a/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:57.232382 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cb770-421f-4882-8d28-02bdb98c3016 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:57.232672 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cb770-421f-4882-8d28-02bdb98c3016 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:57.232951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234cb770-421f-4882-8d28-02bdb98c3016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:57.268563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-234cb770-421f-4882-8d28-02bdb98c3016 None None] MFA Rules not processed for user `b4691caf4e5c4bc299885a1380bd087a`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:57.269840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-234cb770-421f-4882-8d28-02bdb98c3016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:57.311409 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab2276-fe43-44ec-be8a-5815fbca8d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:57.312065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab2276-fe43-44ec-be8a-5815fbca8d32 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:57.312352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab2276-fe43-44ec-be8a-5815fbca8d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:57.312633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab2276-fe43-44ec-be8a-5815fbca8d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:57.313755 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab2276-fe43-44ec-be8a-5815fbca8d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:57.315685 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:57.318303 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab2276-fe43-44ec-be8a-5815fbca8d32 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:57.337229 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1122/2209] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:57.341305 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab2276-fe43-44ec-be8a-5815fbca8d32 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:57.354529 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab2276-fe43-44ec-be8a-5815fbca8d32 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:57.360843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd1d1e-9ec6-427a-9371-171e61701638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:57.361481 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd1d1e-9ec6-427a-9371-171e61701638 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:57.361804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd1d1e-9ec6-427a-9371-171e61701638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:57.362018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd1d1e-9ec6-427a-9371-171e61701638 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:57.362768 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd1d1e-9ec6-427a-9371-171e61701638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:57.363088 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1088/2210] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:54:57.368365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd1d1e-9ec6-427a-9371-171e61701638 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:57.370021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cb770-421f-4882-8d28-02bdb98c3016 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:57.382727 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1123/2211] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:57.384364 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234cb770-421f-4882-8d28-02bdb98c3016 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:57.387337 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd1d1e-9ec6-427a-9371-171e61701638 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:57.392655 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7424d92-ae7a-4f94-8e08-93ce12cca35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:57.393504 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7424d92-ae7a-4f94-8e08-93ce12cca35c admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:57.394075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7424d92-ae7a-4f94-8e08-93ce12cca35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:57.394477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7424d92-ae7a-4f94-8e08-93ce12cca35c admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:57.395163 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7424d92-ae7a-4f94-8e08-93ce12cca35c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:57.397205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7424d92-ae7a-4f94-8e08-93ce12cca35c admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:57.402190 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd1d1e-9ec6-427a-9371-171e61701638 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:57.409314 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7abeb4a-ef17-405f-8b0f-5af21bee413c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:57.410974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abeb4a-ef17-405f-8b0f-5af21bee413c admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:57.411583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abeb4a-ef17-405f-8b0f-5af21bee413c admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:57.412082 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7abeb4a-ef17-405f-8b0f-5af21bee413c admin admin] PATH_INFO: `/v3/projects/3538bb6aa87c437ea7530e53386b1732/users/df68e00df058423d8ff25badc2ce5c06/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:57.431173 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:57.441130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7abeb4a-ef17-405f-8b0f-5af21bee413c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3538bb6aa87c437ea7530e53386b1732, user_id=df68e00df058423d8ff25badc2ce5c06, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:57.442654 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1089/2212] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:57 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:57.444045 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7424d92-ae7a-4f94-8e08-93ce12cca35c admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:57.445606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7abeb4a-ef17-405f-8b0f-5af21bee413c admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:57.472017 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1124/2213] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:57 2022] PUT /identity/v3/projects/3538bb6aa87c437ea7530e53386b1732/users/df68e00df058423d8ff25badc2ce5c06/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:57.485771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71926740-b619-48dc-82b0-913585d533a5 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:57.486163 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71926740-b619-48dc-82b0-913585d533a5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:57.489349 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71926740-b619-48dc-82b0-913585d533a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:57.531554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7424d92-ae7a-4f94-8e08-93ce12cca35c admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:57.553017 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-71926740-b619-48dc-82b0-913585d533a5 None None] MFA Rules not processed for user `df68e00df058423d8ff25badc2ce5c06`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:57.556467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-71926740-b619-48dc-82b0-913585d533a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:57.637131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9265fdc-45c2-4e09-9cc4-07f65ea67242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:57.639884 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9265fdc-45c2-4e09-9cc4-07f65ea67242 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:57.640176 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9265fdc-45c2-4e09-9cc4-07f65ea67242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:57.640477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9265fdc-45c2-4e09-9cc4-07f65ea67242 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:57.641166 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9265fdc-45c2-4e09-9cc4-07f65ea67242 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:57.644702 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9265fdc-45c2-4e09-9cc4-07f65ea67242 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:57.680503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1090/2214] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:57 2022] POST /identity/v3/users => generated 299 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:57.685783 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9265fdc-45c2-4e09-9cc4-07f65ea67242 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:57.695317 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1125/2215] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:57.698305 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9265fdc-45c2-4e09-9cc4-07f65ea67242 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:57.703315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c634d1e-5e96-4e86-9b2f-8a08b266d398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:57.704011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c634d1e-5e96-4e86-9b2f-8a08b266d398 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:57.704400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c634d1e-5e96-4e86-9b2f-8a08b266d398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:57.704751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c634d1e-5e96-4e86-9b2f-8a08b266d398 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:57.705525 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c634d1e-5e96-4e86-9b2f-8a08b266d398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:57.707974 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c634d1e-5e96-4e86-9b2f-8a08b266d398 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:57.724356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1091/2216] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:54:57.727625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71926740-b619-48dc-82b0-913585d533a5 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:57.737565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c634d1e-5e96-4e86-9b2f-8a08b266d398 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:57.740098 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-0c634d1e-5e96-4e86-9b2f-8a08b266d398 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:57.746851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71926740-b619-48dc-82b0-913585d533a5 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:57.765053 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87faa3-6400-4963-be7f-18cbc07a5ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:57.765792 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87faa3-6400-4963-be7f-18cbc07a5ab3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:57.766076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87faa3-6400-4963-be7f-18cbc07a5ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:57.766351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87faa3-6400-4963-be7f-18cbc07a5ab3 admin admin] PATH_INFO: `/v3/projects/a769b0191a0c4175a18965c81019c915/users/529c16638f874168907eacfcda21279d/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:57.807975 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87faa3-6400-4963-be7f-18cbc07a5ab3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a769b0191a0c4175a18965c81019c915, user_id=529c16638f874168907eacfcda21279d, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:57.807975 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87faa3-6400-4963-be7f-18cbc07a5ab3 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:57.831990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1126/2217] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:57 2022] PUT /identity/v3/projects/a769b0191a0c4175a18965c81019c915/users/529c16638f874168907eacfcda21279d/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:57.836272 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87faa3-6400-4963-be7f-18cbc07a5ab3 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:57.839308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:57.882702 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c634d1e-5e96-4e86-9b2f-8a08b266d398 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:57.961264 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04becfaa-d320-4ef0-b87a-faf77b6307ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:57.962258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04becfaa-d320-4ef0-b87a-faf77b6307ee service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:57.962570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04becfaa-d320-4ef0-b87a-faf77b6307ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:57.962878 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04becfaa-d320-4ef0-b87a-faf77b6307ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:57.970191 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87faa3-6400-4963-be7f-18cbc07a5ab3 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:57.975656 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c71f25-c9ba-4cf8-b4e0-e778c0955adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:57.976326 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c71f25-c9ba-4cf8-b4e0-e778c0955adc admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:57.976607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c71f25-c9ba-4cf8-b4e0-e778c0955adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:57.976875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c71f25-c9ba-4cf8-b4e0-e778c0955adc admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:57.977571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c71f25-c9ba-4cf8-b4e0-e778c0955adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:57.979225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c71f25-c9ba-4cf8-b4e0-e778c0955adc admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:57.993445 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1127/2218] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:54:57.997477 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c71f25-c9ba-4cf8-b4e0-e778c0955adc admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:58.011170 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c71f25-c9ba-4cf8-b4e0-e778c0955adc admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:58.024123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c63d3f-662f-476d-9275-7519b53dfcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:58.025105 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c63d3f-662f-476d-9275-7519b53dfcbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:58.025390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c63d3f-662f-476d-9275-7519b53dfcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:58.025667 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c63d3f-662f-476d-9275-7519b53dfcbb admin admin] PATH_INFO: `/v3/projects/a769b0191a0c4175a18965c81019c915/users/529c16638f874168907eacfcda21279d/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:58.028329 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c63d3f-662f-476d-9275-7519b53dfcbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a769b0191a0c4175a18965c81019c915, user_id=529c16638f874168907eacfcda21279d, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:58.031611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c63d3f-662f-476d-9275-7519b53dfcbb admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:58.048336 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1128/2219] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:54:57 2022] PUT /identity/v3/projects/a769b0191a0c4175a18965c81019c915/users/529c16638f874168907eacfcda21279d/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:54:58.054057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04becfaa-d320-4ef0-b87a-faf77b6307ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:58.055562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04becfaa-d320-4ef0-b87a-faf77b6307ee service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:58.060835 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c0d50-1393-492e-b670-c0d42659bc25 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:58.060994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c0d50-1393-492e-b670-c0d42659bc25 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:58.061269 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c0d50-1393-492e-b670-c0d42659bc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:58.067933 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1092/2220] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:57 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:54:58.083835 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04becfaa-d320-4ef0-b87a-faf77b6307ee service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:58.099990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04becfaa-d320-4ef0-b87a-faf77b6307ee service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:58.101438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-563c0d50-1393-492e-b670-c0d42659bc25 None None] MFA Rules not processed for user `529c16638f874168907eacfcda21279d`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:58.102890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-563c0d50-1393-492e-b670-c0d42659bc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:58.134031 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:58.181892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9546a-c274-4d59-bac9-a90be06ea097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:58.182717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9546a-c274-4d59-bac9-a90be06ea097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:58.182994 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9546a-c274-4d59-bac9-a90be06ea097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:58.183320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9546a-c274-4d59-bac9-a90be06ea097 admin admin] PATH_INFO: `/v3/users/df68e00df058423d8ff25badc2ce5c06` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:58.184810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9546a-c274-4d59-bac9-a90be06ea097 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df68e00df058423d8ff25badc2ce5c06)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:58.187064 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9546a-c274-4d59-bac9-a90be06ea097 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:58.199645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1129/2221] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:54:58.213277 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c0d50-1393-492e-b670-c0d42659bc25 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:58.214764 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-563c0d50-1393-492e-b670-c0d42659bc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:58.274906 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-29a9546a-c274-4d59-bac9-a90be06ea097 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df68e00df058423d8ff25badc2ce5c06'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:58.284442 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:58.290258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c0d50-1393-492e-b670-c0d42659bc25 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:58.294158 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0214dc-1b6e-4967-b0e9-92157048c41f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:58.294843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0214dc-1b6e-4967-b0e9-92157048c41f service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:58.295149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0214dc-1b6e-4967-b0e9-92157048c41f service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:58.295428 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0214dc-1b6e-4967-b0e9-92157048c41f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:58.300972 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-29a9546a-c274-4d59-bac9-a90be06ea097 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df68e00df058423d8ff25badc2ce5c06'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:58.311864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-29a9546a-c274-4d59-bac9-a90be06ea097 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df68e00df058423d8ff25badc2ce5c06'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:58.312865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-29a9546a-c274-4d59-bac9-a90be06ea097 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df68e00df058423d8ff25badc2ce5c06'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:58.325824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-29a9546a-c274-4d59-bac9-a90be06ea097 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df68e00df058423d8ff25badc2ce5c06'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:58.338369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1093/2222] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:54:58 2022] DELETE /identity/v3/users/df68e00df058423d8ff25badc2ce5c06 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:58.343171 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de007631-2470-4253-92d1-1127f0bbe363 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:58.343562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de007631-2470-4253-92d1-1127f0bbe363 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:58.345974 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de007631-2470-4253-92d1-1127f0bbe363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:58.371424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0214dc-1b6e-4967-b0e9-92157048c41f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:58.372872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0214dc-1b6e-4967-b0e9-92157048c41f service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:58.383960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1130/2223] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:58.432719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-de007631-2470-4253-92d1-1127f0bbe363 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:58.433978 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-de007631-2470-4253-92d1-1127f0bbe363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:58.643209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1094/2224] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:54:58.651219 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0214dc-1b6e-4967-b0e9-92157048c41f service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:58.653061 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8f0214dc-1b6e-4967-b0e9-92157048c41f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:58.811916 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0214dc-1b6e-4967-b0e9-92157048c41f service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:58.913818 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:54:58.994842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa5278-0457-4767-accb-703d1d5eed9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:58.995834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa5278-0457-4767-accb-703d1d5eed9a service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:58.996128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa5278-0457-4767-accb-703d1d5eed9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:58.996407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa5278-0457-4767-accb-703d1d5eed9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:59.013825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa5278-0457-4767-accb-703d1d5eed9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:59.016651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa5278-0457-4767-accb-703d1d5eed9a service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:59.043508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1131/2225] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:54:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:54:59.505261 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de007631-2470-4253-92d1-1127f0bbe363 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:59.506731 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-de007631-2470-4253-92d1-1127f0bbe363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:59.520938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de007631-2470-4253-92d1-1127f0bbe363 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:59.527007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850f633-60e4-4dd0-a5ab-30d8ff2640f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:59.527770 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850f633-60e4-4dd0-a5ab-30d8ff2640f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:59.527770 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850f633-60e4-4dd0-a5ab-30d8ff2640f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:59.527999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850f633-60e4-4dd0-a5ab-30d8ff2640f2 admin admin] PATH_INFO: `/v3/projects/3538bb6aa87c437ea7530e53386b1732` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:59.529074 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850f633-60e4-4dd0-a5ab-30d8ff2640f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3538bb6aa87c437ea7530e53386b1732)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:59.531028 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850f633-60e4-4dd0-a5ab-30d8ff2640f2 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:59.590808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c850f633-60e4-4dd0-a5ab-30d8ff2640f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3538bb6aa87c437ea7530e53386b1732', 'request_id': 'req-c850f633-60e4-4dd0-a5ab-30d8ff2640f2'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:59.600967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c850f633-60e4-4dd0-a5ab-30d8ff2640f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3538bb6aa87c437ea7530e53386b1732', 'request_id': 'req-c850f633-60e4-4dd0-a5ab-30d8ff2640f2'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:59.619872 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c850f633-60e4-4dd0-a5ab-30d8ff2640f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3538bb6aa87c437ea7530e53386b1732', 'request_id': 'req-c850f633-60e4-4dd0-a5ab-30d8ff2640f2'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:59.640162 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c850f633-60e4-4dd0-a5ab-30d8ff2640f2 admin admin] The token cache is being invalidate because project 3538bb6aa87c437ea7530e53386b1732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:54:59.640456 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c850f633-60e4-4dd0-a5ab-30d8ff2640f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:59.641410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c850f633-60e4-4dd0-a5ab-30d8ff2640f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:54:59.646536 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1095/2226] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:54:59 2022] DELETE /identity/v3/projects/3538bb6aa87c437ea7530e53386b1732 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:54:59.649872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fb5e2-c442-4d9a-9cf0-9f17fe3fba98 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:59.650384 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fb5e2-c442-4d9a-9cf0-9f17fe3fba98 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:59.650519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fb5e2-c442-4d9a-9cf0-9f17fe3fba98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:59.724163 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-e01fb5e2-c442-4d9a-9cf0-9f17fe3fba98 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:59.729635 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e01fb5e2-c442-4d9a-9cf0-9f17fe3fba98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:59.819533 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1132/2227] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:59.827872 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8aa1b0-3c71-4f1c-86ed-b763467dd046 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:59.830270 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8aa1b0-3c71-4f1c-86ed-b763467dd046 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:59.830595 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8aa1b0-3c71-4f1c-86ed-b763467dd046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:59.862559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-2b8aa1b0-3c71-4f1c-86ed-b763467dd046 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:54:59.864924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2b8aa1b0-3c71-4f1c-86ed-b763467dd046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:54:59.874520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1096/2228] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:54:59.878331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fb5e2-c442-4d9a-9cf0-9f17fe3fba98 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:59.890065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fb5e2-c442-4d9a-9cf0-9f17fe3fba98 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:59.895476 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cd20d-c78c-458f-8345-4c44ac782dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:59.896462 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cd20d-c78c-458f-8345-4c44ac782dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:59.897276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cd20d-c78c-458f-8345-4c44ac782dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:59.898044 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749cd20d-c78c-458f-8345-4c44ac782dbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:59.899057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cd20d-c78c-458f-8345-4c44ac782dbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:54:59.899590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cd20d-c78c-458f-8345-4c44ac782dbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:59.901528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749cd20d-c78c-458f-8345-4c44ac782dbb admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:59.917589 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1133/2229] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 12:54:59.922258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8aa1b0-3c71-4f1c-86ed-b763467dd046 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:54:59.933789 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8aa1b0-3c71-4f1c-86ed-b763467dd046 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:54:59.939835 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5af81-8b7e-4047-8b83-703951848e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:54:59.942136 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5af81-8b7e-4047-8b83-703951848e79 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:54:59.942432 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5af81-8b7e-4047-8b83-703951848e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:54:59.942751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5af81-8b7e-4047-8b83-703951848e79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:54:59.943447 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5af81-8b7e-4047-8b83-703951848e79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:54:59.945706 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5af81-8b7e-4047-8b83-703951848e79 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:54:59.989509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1097/2230] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:54:59 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:54:59.993049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cd20d-c78c-458f-8345-4c44ac782dbb admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:00.014511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749cd20d-c78c-458f-8345-4c44ac782dbb admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:00.153555 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73600b18-291d-43b7-ab6b-71403e10d8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:00.155981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73600b18-291d-43b7-ab6b-71403e10d8f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:00.157184 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73600b18-291d-43b7-ab6b-71403e10d8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:00.158098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73600b18-291d-43b7-ab6b-71403e10d8f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:00.159863 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73600b18-291d-43b7-ab6b-71403e10d8f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:00.163055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73600b18-291d-43b7-ab6b-71403e10d8f1 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:00.202846 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1134/2231] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:54:59 2022] POST /identity/v3/users => generated 288 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:55:00.203994 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5af81-8b7e-4047-8b83-703951848e79 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:00.216128 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5af81-8b7e-4047-8b83-703951848e79 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:00.223300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5bcf0-08c9-4b51-a7c3-4710c4bab089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:00.223942 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5bcf0-08c9-4b51-a7c3-4710c4bab089 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:00.224221 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5bcf0-08c9-4b51-a7c3-4710c4bab089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:00.224494 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5bcf0-08c9-4b51-a7c3-4710c4bab089 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:00.225204 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5bcf0-08c9-4b51-a7c3-4710c4bab089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:00.226843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b5bcf0-08c9-4b51-a7c3-4710c4bab089 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:00.281518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1098/2232] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:55:00.285441 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73600b18-291d-43b7-ab6b-71403e10d8f1 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:00.306379 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73600b18-291d-43b7-ab6b-71403e10d8f1 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:00.322157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c20a5e-0537-4dbf-b997-78f7c2e9ee9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:00.325056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c20a5e-0537-4dbf-b997-78f7c2e9ee9f admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:00.325366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c20a5e-0537-4dbf-b997-78f7c2e9ee9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:00.330440 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c20a5e-0537-4dbf-b997-78f7c2e9ee9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:00.331758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c20a5e-0537-4dbf-b997-78f7c2e9ee9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:00.341014 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c20a5e-0537-4dbf-b997-78f7c2e9ee9f admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:00.378194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1135/2233] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:55:00 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:55:00.382657 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5bcf0-08c9-4b51-a7c3-4710c4bab089 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:00.396768 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5bcf0-08c9-4b51-a7c3-4710c4bab089 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:00.403086 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533dc7da-9c85-4412-82f4-b11eb9e9857a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:00.403998 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533dc7da-9c85-4412-82f4-b11eb9e9857a admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:00.404437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533dc7da-9c85-4412-82f4-b11eb9e9857a admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:00.405233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533dc7da-9c85-4412-82f4-b11eb9e9857a admin admin] PATH_INFO: `/v3/projects/4605dd38299c4d209d4e683585c951ae/users/06cdcb5db26644d8bee259bbd8940b36/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:00.421368 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:00.429437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533dc7da-9c85-4412-82f4-b11eb9e9857a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4605dd38299c4d209d4e683585c951ae, user_id=06cdcb5db26644d8bee259bbd8940b36, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:00.433120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533dc7da-9c85-4412-82f4-b11eb9e9857a admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:00.471199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1099/2234] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:55:00 2022] PUT /identity/v3/projects/4605dd38299c4d209d4e683585c951ae/users/06cdcb5db26644d8bee259bbd8940b36/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:55:00.481498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d5ed1-6c66-4091-8cd3-7749c43d173a None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:00.482704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d5ed1-6c66-4091-8cd3-7749c43d173a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:00.482704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d5ed1-6c66-4091-8cd3-7749c43d173a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:00.551952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-182d5ed1-6c66-4091-8cd3-7749c43d173a None None] MFA Rules not processed for user `06cdcb5db26644d8bee259bbd8940b36`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:00.554799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-182d5ed1-6c66-4091-8cd3-7749c43d173a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:00.641159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:00.750909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1136/2235] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:00.755939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533dc7da-9c85-4412-82f4-b11eb9e9857a admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:00.888321 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b71d7-ae4b-452c-99f2-8c798d849083 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:00.888679 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b71d7-ae4b-452c-99f2-8c798d849083 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:00.889247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b71d7-ae4b-452c-99f2-8c798d849083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:00.946919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-088b71d7-ae4b-452c-99f2-8c798d849083 None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:00.948777 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-088b71d7-ae4b-452c-99f2-8c798d849083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:00.950365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533dc7da-9c85-4412-82f4-b11eb9e9857a admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:00.965081 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b2bdf-1d79-4b25-ba09-2f7e836f71ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:00.971333 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b2bdf-1d79-4b25-ba09-2f7e836f71ea admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:00.982501 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b2bdf-1d79-4b25-ba09-2f7e836f71ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:00.982501 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b2bdf-1d79-4b25-ba09-2f7e836f71ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:00.982501 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b2bdf-1d79-4b25-ba09-2f7e836f71ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:00.982501 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b2bdf-1d79-4b25-ba09-2f7e836f71ea admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:01.057283 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1100/2236] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:55:00 2022] POST /identity/v3/projects => generated 373 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:55:01.058975 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b2bdf-1d79-4b25-ba09-2f7e836f71ea admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:01.101493 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1137/2237] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 3 12:55:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:01.165519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b71d7-ae4b-452c-99f2-8c798d849083 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:01.167030 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-088b71d7-ae4b-452c-99f2-8c798d849083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:01.267312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b2bdf-1d79-4b25-ba09-2f7e836f71ea admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:01.276639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae8821-4223-4086-86ea-f9b95f34f997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:01.277290 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae8821-4223-4086-86ea-f9b95f34f997 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:01.277576 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae8821-4223-4086-86ea-f9b95f34f997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:01.277853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae8821-4223-4086-86ea-f9b95f34f997 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:01.278593 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae8821-4223-4086-86ea-f9b95f34f997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:01.280513 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae8821-4223-4086-86ea-f9b95f34f997 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:01.309282 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b71d7-ae4b-452c-99f2-8c798d849083 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:01.328252 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1101/2238] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:55:01 2022] POST /identity/v3/users => generated 287 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:55:01.332941 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae8821-4223-4086-86ea-f9b95f34f997 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:01.357486 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae8821-4223-4086-86ea-f9b95f34f997 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:01.370888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f8032-fd88-4f4a-8c10-69f4b867ccf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:01.372911 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f8032-fd88-4f4a-8c10-69f4b867ccf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:01.373349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f8032-fd88-4f4a-8c10-69f4b867ccf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:01.374024 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f8032-fd88-4f4a-8c10-69f4b867ccf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:01.376960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f8032-fd88-4f4a-8c10-69f4b867ccf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:01.383087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f8032-fd88-4f4a-8c10-69f4b867ccf7 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:01.415452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1102/2239] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:55:01.416603 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f8032-fd88-4f4a-8c10-69f4b867ccf7 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:01.439932 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f8032-fd88-4f4a-8c10-69f4b867ccf7 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:01.446384 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37385559-b3f7-4d3f-9d85-b41d883fc7d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:01.447714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37385559-b3f7-4d3f-9d85-b41d883fc7d2 service swift] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:01.448177 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37385559-b3f7-4d3f-9d85-b41d883fc7d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:01.448676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37385559-b3f7-4d3f-9d85-b41d883fc7d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:01.449324 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f67ae3-f067-4bc2-91a0-d598169ed5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:01.450058 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f67ae3-f067-4bc2-91a0-d598169ed5bf admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:01.450571 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f67ae3-f067-4bc2-91a0-d598169ed5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:01.450954 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f67ae3-f067-4bc2-91a0-d598169ed5bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:01.451760 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f67ae3-f067-4bc2-91a0-d598169ed5bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:01.453469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f67ae3-f067-4bc2-91a0-d598169ed5bf admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:01.478323 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1103/2240] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:55:01.482417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f67ae3-f067-4bc2-91a0-d598169ed5bf admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:01.489132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:01.498516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f67ae3-f067-4bc2-91a0-d598169ed5bf admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:01.511796 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5b4e8-1473-4bc7-8db0-6bb55efdf24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:01.512640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5b4e8-1473-4bc7-8db0-6bb55efdf24c admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:01.514090 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5b4e8-1473-4bc7-8db0-6bb55efdf24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:01.516318 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5b4e8-1473-4bc7-8db0-6bb55efdf24c admin admin] PATH_INFO: `/v3/projects/7425ea807a0e49b7a6d2851892fefdb3/users/d8c171237c664534a31d19c18adb4fef/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:01.567587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5b4e8-1473-4bc7-8db0-6bb55efdf24c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7425ea807a0e49b7a6d2851892fefdb3, user_id=d8c171237c664534a31d19c18adb4fef, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:01.573912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5b4e8-1473-4bc7-8db0-6bb55efdf24c admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:01.575251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37385559-b3f7-4d3f-9d85-b41d883fc7d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:01.576911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37385559-b3f7-4d3f-9d85-b41d883fc7d2 service swift] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:01.586414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1138/2241] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:55:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 03 12:55:01.614250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1104/2242] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:55:01 2022] PUT /identity/v3/projects/7425ea807a0e49b7a6d2851892fefdb3/users/d8c171237c664534a31d19c18adb4fef/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:55:01.635891 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcff819-84dc-4f4e-9131-55c3be209ba7 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:01.636591 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcff819-84dc-4f4e-9131-55c3be209ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:01.636905 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcff819-84dc-4f4e-9131-55c3be209ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:01.693017 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-0fcff819-84dc-4f4e-9131-55c3be209ba7 None None] MFA Rules not processed for user `d8c171237c664534a31d19c18adb4fef`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:01.701816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0fcff819-84dc-4f4e-9131-55c3be209ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:01.850435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1139/2243] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:01.855615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5b4e8-1473-4bc7-8db0-6bb55efdf24c admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:01.912998 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:02.022631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5b4e8-1473-4bc7-8db0-6bb55efdf24c admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:02.049124 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f962a-0d54-4bd4-bd0f-46f0c6b0ad84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:02.049454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f962a-0d54-4bd4-bd0f-46f0c6b0ad84 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:02.049740 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f962a-0d54-4bd4-bd0f-46f0c6b0ad84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:02.050018 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f962a-0d54-4bd4-bd0f-46f0c6b0ad84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:02.051114 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f962a-0d54-4bd4-bd0f-46f0c6b0ad84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:02.053075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f962a-0d54-4bd4-bd0f-46f0c6b0ad84 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:02.111783 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1105/2244] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:55:01 2022] POST /identity/v3/projects => generated 375 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:55:02.116280 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcff819-84dc-4f4e-9131-55c3be209ba7 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:02.136849 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcff819-84dc-4f4e-9131-55c3be209ba7 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:02.261201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd2403-9c92-4a87-9582-5e0b0b1d7036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:02.262243 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd2403-9c92-4a87-9582-5e0b0b1d7036 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:02.262766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd2403-9c92-4a87-9582-5e0b0b1d7036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:02.263264 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd2403-9c92-4a87-9582-5e0b0b1d7036 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:02.264156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd2403-9c92-4a87-9582-5e0b0b1d7036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:02.270020 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cd2403-9c92-4a87-9582-5e0b0b1d7036 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:02.337243 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1140/2245] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:55:02 2022] POST /identity/v3/users => generated 294 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:55:02.339956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f962a-0d54-4bd4-bd0f-46f0c6b0ad84 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:02.369136 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f962a-0d54-4bd4-bd0f-46f0c6b0ad84 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:02.383942 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d52d78-ceda-49e8-8ecc-a520676a6422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:02.387796 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d52d78-ceda-49e8-8ecc-a520676a6422 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:02.388132 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d52d78-ceda-49e8-8ecc-a520676a6422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:02.388511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d52d78-ceda-49e8-8ecc-a520676a6422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:02.389356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d52d78-ceda-49e8-8ecc-a520676a6422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:02.391789 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d52d78-ceda-49e8-8ecc-a520676a6422 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:02.419649 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1106/2246] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:55:02.420960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd2403-9c92-4a87-9582-5e0b0b1d7036 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:02.451049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd2403-9c92-4a87-9582-5e0b0b1d7036 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:02.458276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23eeee-0bdd-42ef-aa68-d6b1b5f4453a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:02.459633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23eeee-0bdd-42ef-aa68-d6b1b5f4453a admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:02.460051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23eeee-0bdd-42ef-aa68-d6b1b5f4453a admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:02.460730 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23eeee-0bdd-42ef-aa68-d6b1b5f4453a admin admin] PATH_INFO: `/v3/projects/27eba69e7998462b86c035eb1468762b/users/f50370ef0072465286d39cf6d1245d7d/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:02.496201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e23eeee-0bdd-42ef-aa68-d6b1b5f4453a admin admin] RBAC: Authorizing `identity:create_grant(project_id=27eba69e7998462b86c035eb1468762b, user_id=f50370ef0072465286d39cf6d1245d7d, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:02.501396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e23eeee-0bdd-42ef-aa68-d6b1b5f4453a admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:02.557799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1141/2247] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:55:02 2022] PUT /identity/v3/projects/27eba69e7998462b86c035eb1468762b/users/f50370ef0072465286d39cf6d1245d7d/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:55:02.563422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d52d78-ceda-49e8-8ecc-a520676a6422 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:02.575953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d52d78-ceda-49e8-8ecc-a520676a6422 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:02.728507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e1581-955c-4788-ad83-c7012666e788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:02.735027 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e1581-955c-4788-ad83-c7012666e788 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:02.735901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e1581-955c-4788-ad83-c7012666e788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:02.738551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e1581-955c-4788-ad83-c7012666e788 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:02.739463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e1581-955c-4788-ad83-c7012666e788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:02.750379 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e1581-955c-4788-ad83-c7012666e788 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:02.794754 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1107/2248] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:55:02.799500 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23eeee-0bdd-42ef-aa68-d6b1b5f4453a admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:02.839204 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23eeee-0bdd-42ef-aa68-d6b1b5f4453a admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:02.863693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c78742-a55d-4e80-a900-f1bb751b5d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:02.865171 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c78742-a55d-4e80-a900-f1bb751b5d31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:02.865674 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c78742-a55d-4e80-a900-f1bb751b5d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:02.866802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c78742-a55d-4e80-a900-f1bb751b5d31 admin admin] PATH_INFO: `/v3/projects/27eba69e7998462b86c035eb1468762b/users/f50370ef0072465286d39cf6d1245d7d/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:02.873995 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c78742-a55d-4e80-a900-f1bb751b5d31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27eba69e7998462b86c035eb1468762b, user_id=f50370ef0072465286d39cf6d1245d7d, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:02.877485 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c78742-a55d-4e80-a900-f1bb751b5d31 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:02.903461 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:02.909793 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1142/2249] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:55:02 2022] PUT /identity/v3/projects/27eba69e7998462b86c035eb1468762b/users/f50370ef0072465286d39cf6d1245d7d/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:55:02.922828 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e4216-3c40-4a5a-90fe-dd7a67b5c0cf None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:02.923979 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e4216-3c40-4a5a-90fe-dd7a67b5c0cf None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:02.924995 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e4216-3c40-4a5a-90fe-dd7a67b5c0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:03.011746 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-c43e4216-3c40-4a5a-90fe-dd7a67b5c0cf None None] MFA Rules not processed for user `f50370ef0072465286d39cf6d1245d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:03.016117 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c43e4216-3c40-4a5a-90fe-dd7a67b5c0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:03.239852 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1108/2250] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:03.281363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c78742-a55d-4e80-a900-f1bb751b5d31 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:03.287720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c4c78742-a55d-4e80-a900-f1bb751b5d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:03.487219 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c78742-a55d-4e80-a900-f1bb751b5d31 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:03.491501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e995727-3052-4b73-a919-5e925505d5fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:03.492123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e995727-3052-4b73-a919-5e925505d5fd service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:03.492428 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e995727-3052-4b73-a919-5e925505d5fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:03.492718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e995727-3052-4b73-a919-5e925505d5fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:03.582437 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e995727-3052-4b73-a919-5e925505d5fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:03.584215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e995727-3052-4b73-a919-5e925505d5fd service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:03.613900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1143/2251] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:55:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:55:04.049487 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e4216-3c40-4a5a-90fe-dd7a67b5c0cf None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:04.053006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c43e4216-3c40-4a5a-90fe-dd7a67b5c0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:04.096207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e4216-3c40-4a5a-90fe-dd7a67b5c0cf None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:04.116600 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268e219-aa11-4c38-8aa2-9f5ccee8db75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:04.116600 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268e219-aa11-4c38-8aa2-9f5ccee8db75 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:04.116600 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268e219-aa11-4c38-8aa2-9f5ccee8db75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:04.116600 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268e219-aa11-4c38-8aa2-9f5ccee8db75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:04.117126 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e268e219-aa11-4c38-8aa2-9f5ccee8db75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:04.152485 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:04.176266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7cdb8-a57d-4a35-a84a-56a38819a0e1 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:04.176845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7cdb8-a57d-4a35-a84a-56a38819a0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:04.177300 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7cdb8-a57d-4a35-a84a-56a38819a0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:04.218353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-53b7cdb8-a57d-4a35-a84a-56a38819a0e1 None None] MFA Rules not processed for user `b142cc45cfab477eb8dfb40f817c9f27`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:04.220361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-53b7cdb8-a57d-4a35-a84a-56a38819a0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:04.299846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268e219-aa11-4c38-8aa2-9f5ccee8db75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:04.302334 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268e219-aa11-4c38-8aa2-9f5ccee8db75 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:04.324576 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1109/2252] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:55:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:55:04.341096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1144/2253] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:04.352376 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268e219-aa11-4c38-8aa2-9f5ccee8db75 service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:04.354969 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e268e219-aa11-4c38-8aa2-9f5ccee8db75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:04.527396 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268e219-aa11-4c38-8aa2-9f5ccee8db75 service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:04.546040 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe4238-707e-44fe-aead-3ecd03add2b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:04.547956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe4238-707e-44fe-aead-3ecd03add2b4 service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:04.548722 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe4238-707e-44fe-aead-3ecd03add2b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:04.552010 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fe4238-707e-44fe-aead-3ecd03add2b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:04.554437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-29fe4238-707e-44fe-aead-3ecd03add2b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:04.704641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe4238-707e-44fe-aead-3ecd03add2b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:04.709331 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fe4238-707e-44fe-aead-3ecd03add2b4 service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:04.727767 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1110/2254] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:55:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:55:05.586256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7cdb8-a57d-4a35-a84a-56a38819a0e1 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:05.598749 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7cdb8-a57d-4a35-a84a-56a38819a0e1 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:05.603324 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2d830-2952-4a96-a7fd-81ab427630df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:05.604161 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2d830-2952-4a96-a7fd-81ab427630df service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:05.604599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2d830-2952-4a96-a7fd-81ab427630df service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:05.605048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2d830-2952-4a96-a7fd-81ab427630df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:05.657261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:05.689683 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2d830-2952-4a96-a7fd-81ab427630df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:05.691850 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2d830-2952-4a96-a7fd-81ab427630df service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:05.713913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1145/2255] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:55:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:55:06.795631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe4238-707e-44fe-aead-3ecd03add2b4 service nova] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:06.813506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fe4238-707e-44fe-aead-3ecd03add2b4 service nova] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:06.818371 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918affb-adaa-428f-bc37-80a9e4901825 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:06.819209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918affb-adaa-428f-bc37-80a9e4901825 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:06.819560 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918affb-adaa-428f-bc37-80a9e4901825 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:06.819992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918affb-adaa-428f-bc37-80a9e4901825 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:06.834696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918affb-adaa-428f-bc37-80a9e4901825 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:06.836232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f918affb-adaa-428f-bc37-80a9e4901825 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:06.852076 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1111/2256] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:55:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:55:09.918150 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2d830-2952-4a96-a7fd-81ab427630df service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:09.919638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-4ef2d830-2952-4a96-a7fd-81ab427630df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:10.015168 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2d830-2952-4a96-a7fd-81ab427630df service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:10.019942 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66477afe-d0ee-4d07-a498-f6a866ce9d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:10.020579 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66477afe-d0ee-4d07-a498-f6a866ce9d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:10.020857 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66477afe-d0ee-4d07-a498-f6a866ce9d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:10.022622 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66477afe-d0ee-4d07-a498-f6a866ce9d43 admin admin] PATH_INFO: `/v3/users/b4691caf4e5c4bc299885a1380bd087a` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:10.022622 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66477afe-d0ee-4d07-a498-f6a866ce9d43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4691caf4e5c4bc299885a1380bd087a)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:10.024223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66477afe-d0ee-4d07-a498-f6a866ce9d43 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:10.072037 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-66477afe-d0ee-4d07-a498-f6a866ce9d43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4691caf4e5c4bc299885a1380bd087a'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:10.079082 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-66477afe-d0ee-4d07-a498-f6a866ce9d43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4691caf4e5c4bc299885a1380bd087a'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:10.090529 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-66477afe-d0ee-4d07-a498-f6a866ce9d43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4691caf4e5c4bc299885a1380bd087a'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:10.091191 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-66477afe-d0ee-4d07-a498-f6a866ce9d43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4691caf4e5c4bc299885a1380bd087a'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:10.091791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-66477afe-d0ee-4d07-a498-f6a866ce9d43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4691caf4e5c4bc299885a1380bd087a'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:10.104512 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1146/2257] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:55:09 2022] DELETE /identity/v3/users/b4691caf4e5c4bc299885a1380bd087a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:55:10.109316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6891a1-a2f2-4fe7-97f7-485e2c3e8063 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:10.109835 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6891a1-a2f2-4fe7-97f7-485e2c3e8063 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:10.109937 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6891a1-a2f2-4fe7-97f7-485e2c3e8063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:10.130219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-2a6891a1-a2f2-4fe7-97f7-485e2c3e8063 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:10.131454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2a6891a1-a2f2-4fe7-97f7-485e2c3e8063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:10.164918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:10.212080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1112/2258] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:10.219000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66477afe-d0ee-4d07-a498-f6a866ce9d43 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:10.220579 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-66477afe-d0ee-4d07-a498-f6a866ce9d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:10.279445 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66477afe-d0ee-4d07-a498-f6a866ce9d43 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:10.283418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af6630-4ef9-4a1b-a093-f06ff5ee234b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:10.284025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af6630-4ef9-4a1b-a093-f06ff5ee234b service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:10.284301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af6630-4ef9-4a1b-a093-f06ff5ee234b service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:10.284572 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af6630-4ef9-4a1b-a093-f06ff5ee234b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:10.295039 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af6630-4ef9-4a1b-a093-f06ff5ee234b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:10.296505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af6630-4ef9-4a1b-a093-f06ff5ee234b service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:10.303305 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:10.308753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1147/2259] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:55:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:55:10.612995 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6891a1-a2f2-4fe7-97f7-485e2c3e8063 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:10.614678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2a6891a1-a2f2-4fe7-97f7-485e2c3e8063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:10.639748 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6891a1-a2f2-4fe7-97f7-485e2c3e8063 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:10.653915 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af48073-96fe-41ef-8c03-21145d183a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:10.655363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af48073-96fe-41ef-8c03-21145d183a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:10.655654 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af48073-96fe-41ef-8c03-21145d183a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:10.655940 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af48073-96fe-41ef-8c03-21145d183a33 admin admin] PATH_INFO: `/v3/projects/f8659387819c43868a0718ff342ebda5` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:10.657218 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af48073-96fe-41ef-8c03-21145d183a33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8659387819c43868a0718ff342ebda5)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:10.659310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af48073-96fe-41ef-8c03-21145d183a33 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:10.723372 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1af48073-96fe-41ef-8c03-21145d183a33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8659387819c43868a0718ff342ebda5', 'request_id': 'req-1af48073-96fe-41ef-8c03-21145d183a33'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:10.732216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1af48073-96fe-41ef-8c03-21145d183a33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8659387819c43868a0718ff342ebda5', 'request_id': 'req-1af48073-96fe-41ef-8c03-21145d183a33'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:10.741793 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1af48073-96fe-41ef-8c03-21145d183a33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8659387819c43868a0718ff342ebda5', 'request_id': 'req-1af48073-96fe-41ef-8c03-21145d183a33'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:10.754748 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1af48073-96fe-41ef-8c03-21145d183a33 admin admin] The token cache is being invalidate because project f8659387819c43868a0718ff342ebda5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:55:10.754748 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1af48073-96fe-41ef-8c03-21145d183a33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:10.755414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1af48073-96fe-41ef-8c03-21145d183a33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:10.757125 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1113/2260] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:55:10 2022] DELETE /identity/v3/projects/f8659387819c43868a0718ff342ebda5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:55:10.761047 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af6630-4ef9-4a1b-a093-f06ff5ee234b service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:10.774637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af6630-4ef9-4a1b-a093-f06ff5ee234b service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:10.917216 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254a3f4-12ab-41b0-a105-0dceb5ac28af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:10.918153 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254a3f4-12ab-41b0-a105-0dceb5ac28af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:10.921069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254a3f4-12ab-41b0-a105-0dceb5ac28af admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:10.921627 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254a3f4-12ab-41b0-a105-0dceb5ac28af admin admin] PATH_INFO: `/v3/users/529c16638f874168907eacfcda21279d` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:10.923435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254a3f4-12ab-41b0-a105-0dceb5ac28af admin admin] RBAC: Authorizing `identity:delete_user(user_id=529c16638f874168907eacfcda21279d)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:10.926015 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0254a3f4-12ab-41b0-a105-0dceb5ac28af admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:11.008328 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-0254a3f4-12ab-41b0-a105-0dceb5ac28af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '529c16638f874168907eacfcda21279d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:11.025035 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-0254a3f4-12ab-41b0-a105-0dceb5ac28af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '529c16638f874168907eacfcda21279d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:11.059334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-0254a3f4-12ab-41b0-a105-0dceb5ac28af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '529c16638f874168907eacfcda21279d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:11.068247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-0254a3f4-12ab-41b0-a105-0dceb5ac28af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '529c16638f874168907eacfcda21279d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:11.068247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-0254a3f4-12ab-41b0-a105-0dceb5ac28af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '529c16638f874168907eacfcda21279d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:11.131502 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1148/2261] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:55:10 2022] DELETE /identity/v3/users/529c16638f874168907eacfcda21279d => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:55:11.225296 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af48073-96fe-41ef-8c03-21145d183a33 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:11.227001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1af48073-96fe-41ef-8c03-21145d183a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:11.248373 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af48073-96fe-41ef-8c03-21145d183a33 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:11.389674 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0edd5a-e84b-4e7b-82ff-a1284e54d277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:11.390459 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0edd5a-e84b-4e7b-82ff-a1284e54d277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:11.390867 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0edd5a-e84b-4e7b-82ff-a1284e54d277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:11.391209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0edd5a-e84b-4e7b-82ff-a1284e54d277 admin admin] PATH_INFO: `/v3/users/06cdcb5db26644d8bee259bbd8940b36` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:11.392845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0edd5a-e84b-4e7b-82ff-a1284e54d277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06cdcb5db26644d8bee259bbd8940b36)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:11.396664 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0edd5a-e84b-4e7b-82ff-a1284e54d277 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:11.479840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cc0edd5a-e84b-4e7b-82ff-a1284e54d277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06cdcb5db26644d8bee259bbd8940b36'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:11.498223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:11.500103 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cc0edd5a-e84b-4e7b-82ff-a1284e54d277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06cdcb5db26644d8bee259bbd8940b36'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:11.526596 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cc0edd5a-e84b-4e7b-82ff-a1284e54d277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06cdcb5db26644d8bee259bbd8940b36'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:11.526596 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cc0edd5a-e84b-4e7b-82ff-a1284e54d277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06cdcb5db26644d8bee259bbd8940b36'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:11.536493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cc0edd5a-e84b-4e7b-82ff-a1284e54d277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06cdcb5db26644d8bee259bbd8940b36'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:11.544541 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1114/2262] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:55:11 2022] DELETE /identity/v3/users/06cdcb5db26644d8bee259bbd8940b36 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:55:11.548854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68feacc-65e0-4d76-b077-00090ef4b83a None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:11.549139 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68feacc-65e0-4d76-b077-00090ef4b83a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:11.549408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68feacc-65e0-4d76-b077-00090ef4b83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:11.575757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-b68feacc-65e0-4d76-b077-00090ef4b83a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:11.577337 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b68feacc-65e0-4d76-b077-00090ef4b83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:11.619465 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0edd5a-e84b-4e7b-82ff-a1284e54d277 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:11.620986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-cc0edd5a-e84b-4e7b-82ff-a1284e54d277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:11.660486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1149/2263] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:11.672073 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68feacc-65e0-4d76-b077-00090ef4b83a None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:11.673746 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b68feacc-65e0-4d76-b077-00090ef4b83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:11.714704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:11.741606 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0edd5a-e84b-4e7b-82ff-a1284e54d277 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:11.748478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee2cf01-b4ed-46ca-87bf-3fd714dbd817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:11.749305 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2cf01-b4ed-46ca-87bf-3fd714dbd817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:11.749707 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2cf01-b4ed-46ca-87bf-3fd714dbd817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:11.750147 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee2cf01-b4ed-46ca-87bf-3fd714dbd817 admin admin] PATH_INFO: `/v3/projects/a769b0191a0c4175a18965c81019c915` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:11.751484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee2cf01-b4ed-46ca-87bf-3fd714dbd817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a769b0191a0c4175a18965c81019c915)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:11.754215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee2cf01-b4ed-46ca-87bf-3fd714dbd817 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:11.772320 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68feacc-65e0-4d76-b077-00090ef4b83a None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:11.776108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a1502-b89a-45ed-8216-53f577f6f6e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:11.776829 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a1502-b89a-45ed-8216-53f577f6f6e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:11.777045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a1502-b89a-45ed-8216-53f577f6f6e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:11.777577 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a1502-b89a-45ed-8216-53f577f6f6e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:11.802974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a1502-b89a-45ed-8216-53f577f6f6e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:11.804695 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a1502-b89a-45ed-8216-53f577f6f6e4 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:11.821327 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1150/2264] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:55:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:55:11.842902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-dee2cf01-b4ed-46ca-87bf-3fd714dbd817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a769b0191a0c4175a18965c81019c915', 'request_id': 'req-dee2cf01-b4ed-46ca-87bf-3fd714dbd817'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:11.854668 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-dee2cf01-b4ed-46ca-87bf-3fd714dbd817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a769b0191a0c4175a18965c81019c915', 'request_id': 'req-dee2cf01-b4ed-46ca-87bf-3fd714dbd817'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:11.869680 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-dee2cf01-b4ed-46ca-87bf-3fd714dbd817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a769b0191a0c4175a18965c81019c915', 'request_id': 'req-dee2cf01-b4ed-46ca-87bf-3fd714dbd817'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:11.905854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-dee2cf01-b4ed-46ca-87bf-3fd714dbd817 admin admin] The token cache is being invalidate because project a769b0191a0c4175a18965c81019c915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:55:11.906151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-dee2cf01-b4ed-46ca-87bf-3fd714dbd817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:11.908255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-dee2cf01-b4ed-46ca-87bf-3fd714dbd817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:11.923612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1115/2265] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:55:11 2022] DELETE /identity/v3/projects/a769b0191a0c4175a18965c81019c915 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:55:11.926030 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbd65b-592e-4b66-b941-67494c0d058a None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:11.927159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbd65b-592e-4b66-b941-67494c0d058a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:11.927594 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbd65b-592e-4b66-b941-67494c0d058a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:11.963394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-d4dbd65b-592e-4b66-b941-67494c0d058a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:11.964838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d4dbd65b-592e-4b66-b941-67494c0d058a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:12.078572 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1151/2266] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:12.084205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50016a6-3734-4b70-8d88-3c728db57d35 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:12.084832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50016a6-3734-4b70-8d88-3c728db57d35 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:12.085063 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50016a6-3734-4b70-8d88-3c728db57d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:12.150653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a50016a6-3734-4b70-8d88-3c728db57d35 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:12.156078 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a50016a6-3734-4b70-8d88-3c728db57d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:12.166349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1116/2267] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:12.170547 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbd65b-592e-4b66-b941-67494c0d058a None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:12.196495 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dbd65b-592e-4b66-b941-67494c0d058a None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:12.205066 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd9f5a-5b23-4528-83a5-ca07bd1b1897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:12.205736 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd9f5a-5b23-4528-83a5-ca07bd1b1897 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:12.206010 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd9f5a-5b23-4528-83a5-ca07bd1b1897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:12.206298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd9f5a-5b23-4528-83a5-ca07bd1b1897 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:12.208347 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd9f5a-5b23-4528-83a5-ca07bd1b1897 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:55:12.208776 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd9f5a-5b23-4528-83a5-ca07bd1b1897 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:12.212328 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cd9f5a-5b23-4528-83a5-ca07bd1b1897 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:12.237185 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1152/2268] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:55:12.245950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd9f5a-5b23-4528-83a5-ca07bd1b1897 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:12.246145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50016a6-3734-4b70-8d88-3c728db57d35 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:12.250343 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a50016a6-3734-4b70-8d88-3c728db57d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:12.273654 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50016a6-3734-4b70-8d88-3c728db57d35 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:12.274026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd9f5a-5b23-4528-83a5-ca07bd1b1897 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:12.281714 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efd1b6-6076-4c5e-9390-dd89afec8303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:12.283304 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efd1b6-6076-4c5e-9390-dd89afec8303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:12.285031 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efd1b6-6076-4c5e-9390-dd89afec8303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:12.285470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efd1b6-6076-4c5e-9390-dd89afec8303 admin admin] PATH_INFO: `/v3/projects/4605dd38299c4d209d4e683585c951ae` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:12.286973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8efd1b6-6076-4c5e-9390-dd89afec8303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4605dd38299c4d209d4e683585c951ae)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:12.289579 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8efd1b6-6076-4c5e-9390-dd89afec8303 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:12.301918 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94969abc-8c50-434c-b3cf-6227050ff383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:12.302610 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94969abc-8c50-434c-b3cf-6227050ff383 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:12.302933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94969abc-8c50-434c-b3cf-6227050ff383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:12.303243 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94969abc-8c50-434c-b3cf-6227050ff383 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:12.303926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94969abc-8c50-434c-b3cf-6227050ff383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:12.307108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94969abc-8c50-434c-b3cf-6227050ff383 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:12.380604 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1153/2269] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:55:12 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:55:12.380824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c8efd1b6-6076-4c5e-9390-dd89afec8303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4605dd38299c4d209d4e683585c951ae', 'request_id': 'req-c8efd1b6-6076-4c5e-9390-dd89afec8303'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:12.381785 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94969abc-8c50-434c-b3cf-6227050ff383 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:12.400385 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:12.412839 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c8efd1b6-6076-4c5e-9390-dd89afec8303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4605dd38299c4d209d4e683585c951ae', 'request_id': 'req-c8efd1b6-6076-4c5e-9390-dd89afec8303'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:12.443244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c8efd1b6-6076-4c5e-9390-dd89afec8303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4605dd38299c4d209d4e683585c951ae', 'request_id': 'req-c8efd1b6-6076-4c5e-9390-dd89afec8303'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:12.460570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c8efd1b6-6076-4c5e-9390-dd89afec8303 admin admin] The token cache is being invalidate because project 4605dd38299c4d209d4e683585c951ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:55:12.461104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c8efd1b6-6076-4c5e-9390-dd89afec8303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:12.461857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c8efd1b6-6076-4c5e-9390-dd89afec8303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:12.465096 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1117/2270] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:55:12 2022] DELETE /identity/v3/projects/4605dd38299c4d209d4e683585c951ae => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:55:12.468304 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efd1b6-6076-4c5e-9390-dd89afec8303 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:12.470913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c8efd1b6-6076-4c5e-9390-dd89afec8303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:12.539978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94969abc-8c50-434c-b3cf-6227050ff383 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:12.552961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44427d47-11c9-436a-bdd7-be03e5f35eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:12.554947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44427d47-11c9-436a-bdd7-be03e5f35eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:12.555782 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44427d47-11c9-436a-bdd7-be03e5f35eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:12.556558 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44427d47-11c9-436a-bdd7-be03e5f35eb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:12.557507 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44427d47-11c9-436a-bdd7-be03e5f35eb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:12.559835 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44427d47-11c9-436a-bdd7-be03e5f35eb6 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:12.600222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1154/2271] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:55:12 2022] POST /identity/v3/users => generated 293 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:55:12.604588 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44427d47-11c9-436a-bdd7-be03e5f35eb6 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:12.607417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-44427d47-11c9-436a-bdd7-be03e5f35eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:12.620052 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efd1b6-6076-4c5e-9390-dd89afec8303 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:12.625961 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fcc643-efc6-4746-8b45-f37b2195db3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:12.627287 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fcc643-efc6-4746-8b45-f37b2195db3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:12.627949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fcc643-efc6-4746-8b45-f37b2195db3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:12.629034 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fcc643-efc6-4746-8b45-f37b2195db3a admin admin] PATH_INFO: `/v3/users/d8c171237c664534a31d19c18adb4fef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:12.630716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fcc643-efc6-4746-8b45-f37b2195db3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c171237c664534a31d19c18adb4fef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:12.633220 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fcc643-efc6-4746-8b45-f37b2195db3a admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:12.647251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44427d47-11c9-436a-bdd7-be03e5f35eb6 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:12.656292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f15a3d-6fb5-4820-9df0-98498c8718e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:12.657264 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f15a3d-6fb5-4820-9df0-98498c8718e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:12.657655 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f15a3d-6fb5-4820-9df0-98498c8718e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:12.657941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f15a3d-6fb5-4820-9df0-98498c8718e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:12.659259 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f15a3d-6fb5-4820-9df0-98498c8718e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:12.662848 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f15a3d-6fb5-4820-9df0-98498c8718e2 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:12.684407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1155/2272] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:55:12.685910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f15a3d-6fb5-4820-9df0-98498c8718e2 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:12.699400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-24fcc643-efc6-4746-8b45-f37b2195db3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c171237c664534a31d19c18adb4fef'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:12.701578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f15a3d-6fb5-4820-9df0-98498c8718e2 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:12.708907 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d314f99-d270-44fa-b5a6-d27c0b198654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:12.709931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d314f99-d270-44fa-b5a6-d27c0b198654 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:12.710272 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d314f99-d270-44fa-b5a6-d27c0b198654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:12.710748 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d314f99-d270-44fa-b5a6-d27c0b198654 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:12.711514 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d314f99-d270-44fa-b5a6-d27c0b198654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:12.715082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-24fcc643-efc6-4746-8b45-f37b2195db3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c171237c664534a31d19c18adb4fef'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:12.715907 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d314f99-d270-44fa-b5a6-d27c0b198654 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:12.717794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:12.725423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-24fcc643-efc6-4746-8b45-f37b2195db3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c171237c664534a31d19c18adb4fef'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:12.726723 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-24fcc643-efc6-4746-8b45-f37b2195db3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c171237c664534a31d19c18adb4fef'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:12.746289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-24fcc643-efc6-4746-8b45-f37b2195db3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c171237c664534a31d19c18adb4fef'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:12.748929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1156/2273] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:55:12.750057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d314f99-d270-44fa-b5a6-d27c0b198654 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:12.755267 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-2d314f99-d270-44fa-b5a6-d27c0b198654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:12.759063 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1118/2274] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:55:12 2022] DELETE /identity/v3/users/d8c171237c664534a31d19c18adb4fef => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:55:12.784192 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d314f99-d270-44fa-b5a6-d27c0b198654 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:12.914866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab1c7c-9ac5-4a08-8474-8133d144e7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:12.915564 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab1c7c-9ac5-4a08-8474-8133d144e7ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:12.916018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab1c7c-9ac5-4a08-8474-8133d144e7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:12.916350 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab1c7c-9ac5-4a08-8474-8133d144e7ea admin admin] PATH_INFO: `/v3/projects/d56ffcad26ce4739b18fc429e81fbca4/users/7e2e2caf92174c85b4111a9d1aef04e8/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:12.934605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab1c7c-9ac5-4a08-8474-8133d144e7ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56ffcad26ce4739b18fc429e81fbca4, user_id=7e2e2caf92174c85b4111a9d1aef04e8, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:12.937440 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab1c7c-9ac5-4a08-8474-8133d144e7ea admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:12.953410 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1157/2275] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:55:12 2022] PUT /identity/v3/projects/d56ffcad26ce4739b18fc429e81fbca4/users/7e2e2caf92174c85b4111a9d1aef04e8/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:55:12.965319 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b756f2-36f8-4bf0-87b3-068b0e297f84 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:12.965748 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b756f2-36f8-4bf0-87b3-068b0e297f84 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:12.966189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b756f2-36f8-4bf0-87b3-068b0e297f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:13.018252 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a6b756f2-36f8-4bf0-87b3-068b0e297f84 None None] MFA Rules not processed for user `7e2e2caf92174c85b4111a9d1aef04e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:13.019916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a6b756f2-36f8-4bf0-87b3-068b0e297f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:13.077678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:13.097789 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1119/2276] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:13.110737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab1c7c-9ac5-4a08-8474-8133d144e7ea admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:13.112412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f1ab1c7c-9ac5-4a08-8474-8133d144e7ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:13.192736 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b756f2-36f8-4bf0-87b3-068b0e297f84 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:13.193528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab1c7c-9ac5-4a08-8474-8133d144e7ea admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:13.195564 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a6b756f2-36f8-4bf0-87b3-068b0e297f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:13.197768 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9f479-9747-4b85-9f9f-ebdf8195daac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:13.198465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9f479-9747-4b85-9f9f-ebdf8195daac service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:13.198846 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9f479-9747-4b85-9f9f-ebdf8195daac service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:13.199323 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9f479-9747-4b85-9f9f-ebdf8195daac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:13.210721 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9f479-9747-4b85-9f9f-ebdf8195daac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:13.219187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9f479-9747-4b85-9f9f-ebdf8195daac service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:13.234175 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1158/2277] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:55:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:55:13.253995 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9f479-9747-4b85-9f9f-ebdf8195daac service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:13.292006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b756f2-36f8-4bf0-87b3-068b0e297f84 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:13.297012 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:13.297659 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:13.297944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:13.298219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc admin admin] PATH_INFO: `/v3/projects/7425ea807a0e49b7a6d2851892fefdb3` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:13.299492 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7425ea807a0e49b7a6d2851892fefdb3)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:13.301707 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:13.323206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9f479-9747-4b85-9f9f-ebdf8195daac service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:13.328048 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f48d70-b686-4e43-9857-5bfe18d9a141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:13.328688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f48d70-b686-4e43-9857-5bfe18d9a141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:13.328973 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f48d70-b686-4e43-9857-5bfe18d9a141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:13.329248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f48d70-b686-4e43-9857-5bfe18d9a141 admin admin] PATH_INFO: `/v3/users/7e2e2caf92174c85b4111a9d1aef04e8` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:13.330626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f48d70-b686-4e43-9857-5bfe18d9a141 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e2e2caf92174c85b4111a9d1aef04e8)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:13.333395 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f48d70-b686-4e43-9857-5bfe18d9a141 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:13.357325 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7425ea807a0e49b7a6d2851892fefdb3', 'request_id': 'req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:13.365432 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7425ea807a0e49b7a6d2851892fefdb3', 'request_id': 'req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:13.367427 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:13.379233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7425ea807a0e49b7a6d2851892fefdb3', 'request_id': 'req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:13.395109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc admin admin] The token cache is being invalidate because project 7425ea807a0e49b7a6d2851892fefdb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:55:13.395427 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:13.396287 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:13.401224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1120/2278] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:55:13 2022] DELETE /identity/v3/projects/7425ea807a0e49b7a6d2851892fefdb3 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:55:13.402177 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:13.403621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:13.405376 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-80f48d70-b686-4e43-9857-5bfe18d9a141 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e2e2caf92174c85b4111a9d1aef04e8'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:13.414159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-80f48d70-b686-4e43-9857-5bfe18d9a141 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e2e2caf92174c85b4111a9d1aef04e8'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:13.430452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-80f48d70-b686-4e43-9857-5bfe18d9a141 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e2e2caf92174c85b4111a9d1aef04e8'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:13.431256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-80f48d70-b686-4e43-9857-5bfe18d9a141 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e2e2caf92174c85b4111a9d1aef04e8'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:13.431924 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-80f48d70-b686-4e43-9857-5bfe18d9a141 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e2e2caf92174c85b4111a9d1aef04e8'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:13.447218 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1159/2279] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:55:13 2022] DELETE /identity/v3/users/7e2e2caf92174c85b4111a9d1aef04e8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:55:13.451374 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddba34-093f-4139-a2c1-ab34f78c401a None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:13.451918 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddba34-093f-4139-a2c1-ab34f78c401a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:13.452274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddba34-093f-4139-a2c1-ab34f78c401a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:13.480111 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-b1ddba34-093f-4139-a2c1-ab34f78c401a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:13.481654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b1ddba34-093f-4139-a2c1-ab34f78c401a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:13.547820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc9a8a-ef67-469b-af5b-6c23cfd5aedc admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:13.553989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed93b0-d6b6-4f50-907b-04e7c598c0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:13.554939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed93b0-d6b6-4f50-907b-04e7c598c0ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:13.555370 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed93b0-d6b6-4f50-907b-04e7c598c0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:13.555774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed93b0-d6b6-4f50-907b-04e7c598c0ee admin admin] PATH_INFO: `/v3/users/f50370ef0072465286d39cf6d1245d7d` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:13.557381 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed93b0-d6b6-4f50-907b-04e7c598c0ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=f50370ef0072465286d39cf6d1245d7d)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:13.557763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1160/2280] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:55:13.559945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed93b0-d6b6-4f50-907b-04e7c598c0ee admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:13.566778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddba34-093f-4139-a2c1-ab34f78c401a None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:13.568241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b1ddba34-093f-4139-a2c1-ab34f78c401a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:13.586499 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:13.643407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-31ed93b0-d6b6-4f50-907b-04e7c598c0ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f50370ef0072465286d39cf6d1245d7d'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:13.663205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-31ed93b0-d6b6-4f50-907b-04e7c598c0ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f50370ef0072465286d39cf6d1245d7d'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:13.673206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddba34-093f-4139-a2c1-ab34f78c401a None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:13.678389 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7a927-0d4d-48c6-a2d5-13c4b331c22b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:13.679128 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7a927-0d4d-48c6-a2d5-13c4b331c22b service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:13.679476 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7a927-0d4d-48c6-a2d5-13c4b331c22b service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:13.679819 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7a927-0d4d-48c6-a2d5-13c4b331c22b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:13.683814 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-31ed93b0-d6b6-4f50-907b-04e7c598c0ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f50370ef0072465286d39cf6d1245d7d'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:13.684597 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-31ed93b0-d6b6-4f50-907b-04e7c598c0ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f50370ef0072465286d39cf6d1245d7d'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:13.694297 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-31ed93b0-d6b6-4f50-907b-04e7c598c0ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f50370ef0072465286d39cf6d1245d7d'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:13.700810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1121/2281] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:55:13 2022] DELETE /identity/v3/users/f50370ef0072465286d39cf6d1245d7d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:55:13.707979 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7a927-0d4d-48c6-a2d5-13c4b331c22b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:13.709578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7a927-0d4d-48c6-a2d5-13c4b331c22b service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:13.720460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1161/2282] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:55:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:55:14.120618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed93b0-d6b6-4f50-907b-04e7c598c0ee admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:14.122219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-31ed93b0-d6b6-4f50-907b-04e7c598c0ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:14.230123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7a927-0d4d-48c6-a2d5-13c4b331c22b service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:14.231443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-18f7a927-0d4d-48c6-a2d5-13c4b331c22b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:14.259947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7a927-0d4d-48c6-a2d5-13c4b331c22b service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:14.264950 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed93b0-d6b6-4f50-907b-04e7c598c0ee admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:14.266408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c79023-13ad-4990-bb00-a0dcf08b25fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:14.267206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c79023-13ad-4990-bb00-a0dcf08b25fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:14.267372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c79023-13ad-4990-bb00-a0dcf08b25fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:14.267653 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c79023-13ad-4990-bb00-a0dcf08b25fe admin admin] PATH_INFO: `/v3/projects/d56ffcad26ce4739b18fc429e81fbca4` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:14.270607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c79023-13ad-4990-bb00-a0dcf08b25fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56ffcad26ce4739b18fc429e81fbca4)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:14.272679 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c79023-13ad-4990-bb00-a0dcf08b25fe admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:14.278319 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8214dd-2662-4b9c-b1ba-138dc4c7210e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:14.279715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8214dd-2662-4b9c-b1ba-138dc4c7210e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:14.281744 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8214dd-2662-4b9c-b1ba-138dc4c7210e admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:14.282028 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8214dd-2662-4b9c-b1ba-138dc4c7210e admin admin] PATH_INFO: `/v3/projects/27eba69e7998462b86c035eb1468762b` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:14.285589 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8214dd-2662-4b9c-b1ba-138dc4c7210e admin admin] RBAC: Authorizing `identity:delete_project(project_id=27eba69e7998462b86c035eb1468762b)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:14.287825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8214dd-2662-4b9c-b1ba-138dc4c7210e admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:14.328637 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:14.344295 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-08c79023-13ad-4990-bb00-a0dcf08b25fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56ffcad26ce4739b18fc429e81fbca4', 'request_id': 'req-08c79023-13ad-4990-bb00-a0dcf08b25fe'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:14.357836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-5f8214dd-2662-4b9c-b1ba-138dc4c7210e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27eba69e7998462b86c035eb1468762b', 'request_id': 'req-5f8214dd-2662-4b9c-b1ba-138dc4c7210e'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:14.358468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-08c79023-13ad-4990-bb00-a0dcf08b25fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56ffcad26ce4739b18fc429e81fbca4', 'request_id': 'req-08c79023-13ad-4990-bb00-a0dcf08b25fe'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:14.367104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-5f8214dd-2662-4b9c-b1ba-138dc4c7210e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27eba69e7998462b86c035eb1468762b', 'request_id': 'req-5f8214dd-2662-4b9c-b1ba-138dc4c7210e'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:14.368214 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-08c79023-13ad-4990-bb00-a0dcf08b25fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56ffcad26ce4739b18fc429e81fbca4', 'request_id': 'req-08c79023-13ad-4990-bb00-a0dcf08b25fe'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:14.376113 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-5f8214dd-2662-4b9c-b1ba-138dc4c7210e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27eba69e7998462b86c035eb1468762b', 'request_id': 'req-5f8214dd-2662-4b9c-b1ba-138dc4c7210e'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:14.385138 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-08c79023-13ad-4990-bb00-a0dcf08b25fe admin admin] The token cache is being invalidate because project d56ffcad26ce4739b18fc429e81fbca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:55:14.385431 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-08c79023-13ad-4990-bb00-a0dcf08b25fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:14.386292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-08c79023-13ad-4990-bb00-a0dcf08b25fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:14.392265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1162/2283] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:55:14 2022] DELETE /identity/v3/projects/d56ffcad26ce4739b18fc429e81fbca4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:55:14.394523 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ba428-0505-4acc-80fb-b4f5b50c8bb9 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:14.394834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ba428-0505-4acc-80fb-b4f5b50c8bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:14.395156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ba428-0505-4acc-80fb-b4f5b50c8bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:14.399696 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-5f8214dd-2662-4b9c-b1ba-138dc4c7210e admin admin] The token cache is being invalidate because project 27eba69e7998462b86c035eb1468762b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:55:14.400706 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-5f8214dd-2662-4b9c-b1ba-138dc4c7210e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:14.404097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-5f8214dd-2662-4b9c-b1ba-138dc4c7210e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:14.405967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1122/2284] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:55:14 2022] DELETE /identity/v3/projects/27eba69e7998462b86c035eb1468762b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:55:14.420696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-5c1ba428-0505-4acc-80fb-b4f5b50c8bb9 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:14.421950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5c1ba428-0505-4acc-80fb-b4f5b50c8bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:14.527896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1163/2285] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:14.540026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4a9b6-dcea-47c3-a375-8b51c80c2816 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:14.540352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4a9b6-dcea-47c3-a375-8b51c80c2816 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:14.540983 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4a9b6-dcea-47c3-a375-8b51c80c2816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:14.571392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-61f4a9b6-dcea-47c3-a375-8b51c80c2816 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:14.572750 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-61f4a9b6-dcea-47c3-a375-8b51c80c2816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:14.582755 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1123/2286] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:14.587049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ba428-0505-4acc-80fb-b4f5b50c8bb9 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:14.601592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ba428-0505-4acc-80fb-b4f5b50c8bb9 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:14.607421 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729dc0b4-663b-4f61-9d00-0d79b45603f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:14.608281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729dc0b4-663b-4f61-9d00-0d79b45603f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:14.611636 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729dc0b4-663b-4f61-9d00-0d79b45603f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:14.611636 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729dc0b4-663b-4f61-9d00-0d79b45603f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:14.611636 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729dc0b4-663b-4f61-9d00-0d79b45603f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:55:14.611636 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729dc0b4-663b-4f61-9d00-0d79b45603f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:14.613330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729dc0b4-663b-4f61-9d00-0d79b45603f7 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:14.631790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1164/2287] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:55:14.633169 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4a9b6-dcea-47c3-a375-8b51c80c2816 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:14.647094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4a9b6-dcea-47c3-a375-8b51c80c2816 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:14.653632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd79046-82ce-47a5-82cf-83b39fa36b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:14.654425 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79046-82ce-47a5-82cf-83b39fa36b41 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:14.654850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79046-82ce-47a5-82cf-83b39fa36b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:14.655239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79046-82ce-47a5-82cf-83b39fa36b41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:14.656045 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd79046-82ce-47a5-82cf-83b39fa36b41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:14.658104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd79046-82ce-47a5-82cf-83b39fa36b41 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:14.684356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:14.694236 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1124/2288] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:55:14 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:55:14.696521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729dc0b4-663b-4f61-9d00-0d79b45603f7 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:14.708045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729dc0b4-663b-4f61-9d00-0d79b45603f7 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:14.804799 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe1e80-ad0c-47aa-8350-e4890750dbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:14.805428 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe1e80-ad0c-47aa-8350-e4890750dbfb admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:14.805722 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe1e80-ad0c-47aa-8350-e4890750dbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:14.806098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe1e80-ad0c-47aa-8350-e4890750dbfb admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:14.806808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe1e80-ad0c-47aa-8350-e4890750dbfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:14.808727 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe1e80-ad0c-47aa-8350-e4890750dbfb admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:14.834652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1165/2289] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:55:14 2022] POST /identity/v3/users => generated 284 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:55:14.837027 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd79046-82ce-47a5-82cf-83b39fa36b41 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:14.850664 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd79046-82ce-47a5-82cf-83b39fa36b41 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:14.860753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462dd53-5e9f-477f-be35-2638af43653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:14.861472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462dd53-5e9f-477f-be35-2638af43653a admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:14.861890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462dd53-5e9f-477f-be35-2638af43653a admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:14.862319 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462dd53-5e9f-477f-be35-2638af43653a admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:14.863878 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462dd53-5e9f-477f-be35-2638af43653a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:14.865647 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462dd53-5e9f-477f-be35-2638af43653a admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:14.882040 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1125/2290] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:55:14.883687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe1e80-ad0c-47aa-8350-e4890750dbfb admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:14.896816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe1e80-ad0c-47aa-8350-e4890750dbfb admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:14.901770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55356fbe-90cd-408f-bb10-6a050f3bd50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:14.902592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55356fbe-90cd-408f-bb10-6a050f3bd50a admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:14.903019 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55356fbe-90cd-408f-bb10-6a050f3bd50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:14.903413 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55356fbe-90cd-408f-bb10-6a050f3bd50a admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:14.904233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55356fbe-90cd-408f-bb10-6a050f3bd50a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:14.906357 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55356fbe-90cd-408f-bb10-6a050f3bd50a admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:14.922521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1166/2291] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:55:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:55:14.923739 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462dd53-5e9f-477f-be35-2638af43653a admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:14.943722 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462dd53-5e9f-477f-be35-2638af43653a admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:14.950433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d0fbde-93e2-437e-bb94-7809356f4f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:14.951173 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0fbde-93e2-437e-bb94-7809356f4f09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:14.951464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0fbde-93e2-437e-bb94-7809356f4f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:14.951812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0fbde-93e2-437e-bb94-7809356f4f09 admin admin] PATH_INFO: `/v3/projects/519af30a388145bab2fea77c2b23d191/users/40087e790eaa48a29f4fc2db0a35963a/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:14.967924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d0fbde-93e2-437e-bb94-7809356f4f09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=519af30a388145bab2fea77c2b23d191, user_id=40087e790eaa48a29f4fc2db0a35963a, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:14.970580 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d0fbde-93e2-437e-bb94-7809356f4f09 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:14.986526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1126/2292] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:55:14 2022] PUT /identity/v3/projects/519af30a388145bab2fea77c2b23d191/users/40087e790eaa48a29f4fc2db0a35963a/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:55:14.998508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b14cf-ad00-4bc2-b82e-0b5a1457959d None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:14.998868 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b14cf-ad00-4bc2-b82e-0b5a1457959d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:14.999235 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b14cf-ad00-4bc2-b82e-0b5a1457959d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:15.030018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-c91b14cf-ad00-4bc2-b82e-0b5a1457959d None None] MFA Rules not processed for user `40087e790eaa48a29f4fc2db0a35963a`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:15.031492 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c91b14cf-ad00-4bc2-b82e-0b5a1457959d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:15.056435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:15.117593 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1167/2293] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:24.815742 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d0fbde-93e2-437e-bb94-7809356f4f09 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:24.821754 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c4d0fbde-93e2-437e-bb94-7809356f4f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:24.917356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d0fbde-93e2-437e-bb94-7809356f4f09 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:24.923378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab7e09-4e78-4b9d-9bd5-553f099c22bb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6066e48aa644a47a889e09cfe204858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:24.924704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab7e09-4e78-4b9d-9bd5-553f099c22bb service glance] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:24.925693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab7e09-4e78-4b9d-9bd5-553f099c22bb service glance] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:24.925979 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab7e09-4e78-4b9d-9bd5-553f099c22bb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:24.938709 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab7e09-4e78-4b9d-9bd5-553f099c22bb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:24.941528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab7e09-4e78-4b9d-9bd5-553f099c22bb service glance] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:24.953061 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1127/2294] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:55:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:55:25.113647 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=17 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:31.237258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbe17f-1731-440e-9066-eaf578f54597 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:31.238427 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbe17f-1731-440e-9066-eaf578f54597 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:31.238427 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbe17f-1731-440e-9066-eaf578f54597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:31.257583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-f8bbe17f-1731-440e-9066-eaf578f54597 None None] MFA Rules not processed for user `8ccd455b417549e3b00439cee4577d11`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:31.258816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f8bbe17f-1731-440e-9066-eaf578f54597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:31.321251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1168/2295] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:31.879418 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69118a50-bc8c-4395-8928-ce090952c8db None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:31.879944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69118a50-bc8c-4395-8928-ce090952c8db None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:31.879944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69118a50-bc8c-4395-8928-ce090952c8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:31.908572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-69118a50-bc8c-4395-8928-ce090952c8db None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:31.910533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-69118a50-bc8c-4395-8928-ce090952c8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:31.997659 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1128/2296] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 3 12:55:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:32.032941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbe17f-1731-440e-9066-eaf578f54597 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:32.034722 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f8bbe17f-1731-440e-9066-eaf578f54597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:32.169190 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbe17f-1731-440e-9066-eaf578f54597 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:32.176951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123e904-7f05-48f5-8a14-12bd78cc4f72 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:32.177883 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123e904-7f05-48f5-8a14-12bd78cc4f72 service swift] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:32.178438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123e904-7f05-48f5-8a14-12bd78cc4f72 service swift] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:32.179030 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123e904-7f05-48f5-8a14-12bd78cc4f72 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:32.195875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123e904-7f05-48f5-8a14-12bd78cc4f72 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:32.197602 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123e904-7f05-48f5-8a14-12bd78cc4f72 service swift] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:32.211033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1169/2297] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:55:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 03 12:55:35.118515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69118a50-bc8c-4395-8928-ce090952c8db None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:35.120175 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-69118a50-bc8c-4395-8928-ce090952c8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:35.237508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69118a50-bc8c-4395-8928-ce090952c8db None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:35.244705 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd7157f-1dc3-4243-b1d1-2a73c099c061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:35.245373 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7157f-1dc3-4243-b1d1-2a73c099c061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:35.245656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7157f-1dc3-4243-b1d1-2a73c099c061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:35.245930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd7157f-1dc3-4243-b1d1-2a73c099c061 admin admin] PATH_INFO: `/v3/users/0e8dbb33c0494f01b5066affa7aa74c2` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:35.247253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd7157f-1dc3-4243-b1d1-2a73c099c061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e8dbb33c0494f01b5066affa7aa74c2)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:35.249540 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd7157f-1dc3-4243-b1d1-2a73c099c061 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:35.269831 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:35.321130 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8dd7157f-1dc3-4243-b1d1-2a73c099c061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e8dbb33c0494f01b5066affa7aa74c2'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:35.336222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8dd7157f-1dc3-4243-b1d1-2a73c099c061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e8dbb33c0494f01b5066affa7aa74c2'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:35.344568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8dd7157f-1dc3-4243-b1d1-2a73c099c061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e8dbb33c0494f01b5066affa7aa74c2'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:35.345162 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8dd7157f-1dc3-4243-b1d1-2a73c099c061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e8dbb33c0494f01b5066affa7aa74c2'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:35.354209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8dd7157f-1dc3-4243-b1d1-2a73c099c061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e8dbb33c0494f01b5066affa7aa74c2'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:35.368172 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1129/2298] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:55:35 2022] DELETE /identity/v3/users/0e8dbb33c0494f01b5066affa7aa74c2 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:55:35.898907 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123e904-7f05-48f5-8a14-12bd78cc4f72 service swift] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:35.915633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123e904-7f05-48f5-8a14-12bd78cc4f72 service swift] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:36.014222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0a349-c200-4043-b626-b8bead6041be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:36.014998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0a349-c200-4043-b626-b8bead6041be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:36.015348 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0a349-c200-4043-b626-b8bead6041be admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:36.015701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0a349-c200-4043-b626-b8bead6041be admin admin] PATH_INFO: `/v3/projects/caa81215b6274fb68edc6e3e57419da9` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:36.016972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0a349-c200-4043-b626-b8bead6041be admin admin] RBAC: Authorizing `identity:delete_project(project_id=caa81215b6274fb68edc6e3e57419da9)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:36.018986 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0a349-c200-4043-b626-b8bead6041be admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:36.077939 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:36.094131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-80f0a349-c200-4043-b626-b8bead6041be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caa81215b6274fb68edc6e3e57419da9', 'request_id': 'req-80f0a349-c200-4043-b626-b8bead6041be'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:36.106677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-80f0a349-c200-4043-b626-b8bead6041be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caa81215b6274fb68edc6e3e57419da9', 'request_id': 'req-80f0a349-c200-4043-b626-b8bead6041be'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:36.117415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-80f0a349-c200-4043-b626-b8bead6041be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caa81215b6274fb68edc6e3e57419da9', 'request_id': 'req-80f0a349-c200-4043-b626-b8bead6041be'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:36.131144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-80f0a349-c200-4043-b626-b8bead6041be admin admin] The token cache is being invalidate because project caa81215b6274fb68edc6e3e57419da9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:55:36.131273 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-80f0a349-c200-4043-b626-b8bead6041be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:36.131903 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-80f0a349-c200-4043-b626-b8bead6041be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:55:36.133539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1170/2299] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:55:35 2022] DELETE /identity/v3/projects/caa81215b6274fb68edc6e3e57419da9 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:55:36.138678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124c435-2979-4f26-b98d-aa1874f988b4 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:36.139213 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124c435-2979-4f26-b98d-aa1874f988b4 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:36.139213 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124c435-2979-4f26-b98d-aa1874f988b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:36.159152 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-f124c435-2979-4f26-b98d-aa1874f988b4 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:36.166380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f124c435-2979-4f26-b98d-aa1874f988b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:36.291998 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1130/2300] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:36.300294 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703acc62-73e4-426c-aca8-b457f2d12968 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:36.301361 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703acc62-73e4-426c-aca8-b457f2d12968 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:36.301747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703acc62-73e4-426c-aca8-b457f2d12968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:36.326189 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-703acc62-73e4-426c-aca8-b457f2d12968 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:36.326189 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-703acc62-73e4-426c-aca8-b457f2d12968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:36.336233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1171/2301] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:36.340282 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124c435-2979-4f26-b98d-aa1874f988b4 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:36.352347 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124c435-2979-4f26-b98d-aa1874f988b4 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:36.358865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eccefa2-0e12-4697-afee-f521739c3d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:36.359701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccefa2-0e12-4697-afee-f521739c3d38 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:36.360200 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccefa2-0e12-4697-afee-f521739c3d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:36.360686 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eccefa2-0e12-4697-afee-f521739c3d38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:36.361557 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eccefa2-0e12-4697-afee-f521739c3d38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:55:36.362077 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eccefa2-0e12-4697-afee-f521739c3d38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:36.364363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eccefa2-0e12-4697-afee-f521739c3d38 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:36.379859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1131/2302] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:55:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:55:36.384425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703acc62-73e4-426c-aca8-b457f2d12968 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:36.401050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703acc62-73e4-426c-aca8-b457f2d12968 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:36.407287 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab19556-b749-473d-892c-60fcf9c6de8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:36.408279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab19556-b749-473d-892c-60fcf9c6de8c admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:36.408856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab19556-b749-473d-892c-60fcf9c6de8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:36.409322 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab19556-b749-473d-892c-60fcf9c6de8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:36.419983 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab19556-b749-473d-892c-60fcf9c6de8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:36.421523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b936d-2226-49f6-9ae0-376da3219209 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:36.421813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b936d-2226-49f6-9ae0-376da3219209 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:36.422129 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b936d-2226-49f6-9ae0-376da3219209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:36.422246 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab19556-b749-473d-892c-60fcf9c6de8c admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:36.444310 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-369b936d-2226-49f6-9ae0-376da3219209 None None] MFA Rules not processed for user `44fbd50cb5664149814abf0e9e44e18f`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:36.445541 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-369b936d-2226-49f6-9ae0-376da3219209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:36.470655 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1172/2303] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:55:36 2022] POST /identity/v3/projects => generated 391 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:55:36.471804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab19556-b749-473d-892c-60fcf9c6de8c admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:36.520292 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1132/2304] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:36.550122 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab19556-b749-473d-892c-60fcf9c6de8c admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:36.555621 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d16c3c-bd51-48fc-bf7f-b9e6a7922698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:36.556239 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d16c3c-bd51-48fc-bf7f-b9e6a7922698 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:36.556518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d16c3c-bd51-48fc-bf7f-b9e6a7922698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:36.556786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d16c3c-bd51-48fc-bf7f-b9e6a7922698 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:36.557477 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d16c3c-bd51-48fc-bf7f-b9e6a7922698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:36.559407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d16c3c-bd51-48fc-bf7f-b9e6a7922698 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:36.586552 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1173/2305] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:55:36 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:55:36.588054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b936d-2226-49f6-9ae0-376da3219209 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:36.599313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b936d-2226-49f6-9ae0-376da3219209 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:36.603931 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206da7e8-e164-40e8-b965-407c6fd19fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:36.604541 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206da7e8-e164-40e8-b965-407c6fd19fce admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:36.604818 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206da7e8-e164-40e8-b965-407c6fd19fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:36.605086 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206da7e8-e164-40e8-b965-407c6fd19fce admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:36.605849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206da7e8-e164-40e8-b965-407c6fd19fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:36.607471 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206da7e8-e164-40e8-b965-407c6fd19fce admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:36.618866 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1133/2306] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:55:36.623359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d16c3c-bd51-48fc-bf7f-b9e6a7922698 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:36.634132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d16c3c-bd51-48fc-bf7f-b9e6a7922698 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:36.639594 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c473844-840e-4959-a16c-1f132c59fa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:36.640217 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c473844-840e-4959-a16c-1f132c59fa5a admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:36.640528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c473844-840e-4959-a16c-1f132c59fa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:36.640938 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c473844-840e-4959-a16c-1f132c59fa5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:36.641639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c473844-840e-4959-a16c-1f132c59fa5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:36.643271 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c473844-840e-4959-a16c-1f132c59fa5a admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:36.655164 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1174/2307] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:55:36.659053 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206da7e8-e164-40e8-b965-407c6fd19fce admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:36.672009 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206da7e8-e164-40e8-b965-407c6fd19fce admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:36.678075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6b7a5-74d6-47c2-b930-dd7cbd1c8d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:36.678843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6b7a5-74d6-47c2-b930-dd7cbd1c8d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:36.679137 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6b7a5-74d6-47c2-b930-dd7cbd1c8d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:36.679415 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6b7a5-74d6-47c2-b930-dd7cbd1c8d77 admin admin] PATH_INFO: `/v3/projects/b76cd588e38a4b3393dd2f79c2632266/users/01f5d8ba67c8413083b605a38388b984/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:36.692758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:36.697250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6b7a5-74d6-47c2-b930-dd7cbd1c8d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b76cd588e38a4b3393dd2f79c2632266, user_id=01f5d8ba67c8413083b605a38388b984, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:36.700558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6b7a5-74d6-47c2-b930-dd7cbd1c8d77 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:36.716156 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1134/2308] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:55:36 2022] PUT /identity/v3/projects/b76cd588e38a4b3393dd2f79c2632266/users/01f5d8ba67c8413083b605a38388b984/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:55:36.726503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc22a94-0623-40cc-8daa-91116bc1fcb6 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:36.726794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc22a94-0623-40cc-8daa-91116bc1fcb6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:36.727074 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc22a94-0623-40cc-8daa-91116bc1fcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:36.757562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-3fc22a94-0623-40cc-8daa-91116bc1fcb6 None None] MFA Rules not processed for user `01f5d8ba67c8413083b605a38388b984`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:36.759034 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-3fc22a94-0623-40cc-8daa-91116bc1fcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:36.855397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1175/2309] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:55:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:36.868903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6b7a5-74d6-47c2-b930-dd7cbd1c8d77 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:36.872201 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-4ec6b7a5-74d6-47c2-b930-dd7cbd1c8d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:36.949260 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6b7a5-74d6-47c2-b930-dd7cbd1c8d77 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:36.953608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3862eda-17ee-44a6-8ccf-b4fc87ba049d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:36.954240 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3862eda-17ee-44a6-8ccf-b4fc87ba049d service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:36.954705 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3862eda-17ee-44a6-8ccf-b4fc87ba049d service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:36.954980 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3862eda-17ee-44a6-8ccf-b4fc87ba049d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:36.968617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3862eda-17ee-44a6-8ccf-b4fc87ba049d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:36.970796 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3862eda-17ee-44a6-8ccf-b4fc87ba049d service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:36.985298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1135/2310] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:55:36 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:55:46.851866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:46.979116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:55:56.997368 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e43a5-d8eb-4f4a-b6e4-ebd0fcb621b5 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:56.997873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e43a5-d8eb-4f4a-b6e4-ebd0fcb621b5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:56.997972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e43a5-d8eb-4f4a-b6e4-ebd0fcb621b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:57.019691 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-0d6e43a5-d8eb-4f4a-b6e4-ebd0fcb621b5 None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:55:57.020973 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0d6e43a5-d8eb-4f4a-b6e4-ebd0fcb621b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:57.076210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1176/2311] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 3 12:55:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:55:57.088031 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3862eda-17ee-44a6-8ccf-b4fc87ba049d service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:55:57.089500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c3862eda-17ee-44a6-8ccf-b4fc87ba049d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:55:57.158051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3862eda-17ee-44a6-8ccf-b4fc87ba049d service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:55:57.163523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b470988-e496-469d-979c-c48aecf539cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:55:57.164135 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b470988-e496-469d-979c-c48aecf539cf service swift] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:55:57.164416 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b470988-e496-469d-979c-c48aecf539cf service swift] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:55:57.164691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b470988-e496-469d-979c-c48aecf539cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:55:57.172622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b470988-e496-469d-979c-c48aecf539cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:55:57.174188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b470988-e496-469d-979c-c48aecf539cf service swift] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:55:57.183055 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1136/2312] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:55:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 12:56:00.400172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e43a5-d8eb-4f4a-b6e4-ebd0fcb621b5 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:00.402017 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0d6e43a5-d8eb-4f4a-b6e4-ebd0fcb621b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:00.509315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e43a5-d8eb-4f4a-b6e4-ebd0fcb621b5 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:00.518266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d82853d-6f50-4b53-a3f2-0a776cbc8069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:00.519723 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82853d-6f50-4b53-a3f2-0a776cbc8069 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:00.520154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82853d-6f50-4b53-a3f2-0a776cbc8069 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:00.520596 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d82853d-6f50-4b53-a3f2-0a776cbc8069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:00.524161 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0d82853d-6f50-4b53-a3f2-0a776cbc8069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:00.624559 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82853d-6f50-4b53-a3f2-0a776cbc8069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:00.625988 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d82853d-6f50-4b53-a3f2-0a776cbc8069 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:00.641237 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1177/2313] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:56:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:56:00.887042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b470988-e496-469d-979c-c48aecf539cf service swift] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:01.029056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b470988-e496-469d-979c-c48aecf539cf service swift] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:01.036612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec530731-87a4-4667-a267-760049463860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:01.037282 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec530731-87a4-4667-a267-760049463860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:01.037571 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec530731-87a4-4667-a267-760049463860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:01.037844 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec530731-87a4-4667-a267-760049463860 admin admin] PATH_INFO: `/v3/users/01f5d8ba67c8413083b605a38388b984` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:01.039069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec530731-87a4-4667-a267-760049463860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01f5d8ba67c8413083b605a38388b984)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:01.041461 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec530731-87a4-4667-a267-760049463860 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:01.124095 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ec530731-87a4-4667-a267-760049463860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01f5d8ba67c8413083b605a38388b984'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:01.140363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ec530731-87a4-4667-a267-760049463860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01f5d8ba67c8413083b605a38388b984'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:01.151081 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ec530731-87a4-4667-a267-760049463860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01f5d8ba67c8413083b605a38388b984'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:01.154008 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ec530731-87a4-4667-a267-760049463860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01f5d8ba67c8413083b605a38388b984'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:01.162447 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ec530731-87a4-4667-a267-760049463860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01f5d8ba67c8413083b605a38388b984'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:01.169543 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1137/2314] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:56:00 2022] DELETE /identity/v3/users/01f5d8ba67c8413083b605a38388b984 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:56:01.177260 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e22de6-a5ff-4e92-91cc-8424ceaf5dd9 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:01.177624 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e22de6-a5ff-4e92-91cc-8424ceaf5dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:01.177979 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e22de6-a5ff-4e92-91cc-8424ceaf5dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:01.198750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-51e22de6-a5ff-4e92-91cc-8424ceaf5dd9 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:01.199844 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-51e22de6-a5ff-4e92-91cc-8424ceaf5dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:01.260499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:01.273936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1178/2315] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:01.283767 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec530731-87a4-4667-a267-760049463860 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:01.285661 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ec530731-87a4-4667-a267-760049463860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:01.302656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:01.423140 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec530731-87a4-4667-a267-760049463860 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:01.431940 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea930016-8f5d-4964-a2be-a43a8d677eea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:01.432764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea930016-8f5d-4964-a2be-a43a8d677eea service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:01.433125 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea930016-8f5d-4964-a2be-a43a8d677eea service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:01.433464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea930016-8f5d-4964-a2be-a43a8d677eea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:01.447319 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea930016-8f5d-4964-a2be-a43a8d677eea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:01.448871 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea930016-8f5d-4964-a2be-a43a8d677eea service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:01.465244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1138/2316] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:56:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:56:02.156742 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e22de6-a5ff-4e92-91cc-8424ceaf5dd9 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:02.158320 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-51e22de6-a5ff-4e92-91cc-8424ceaf5dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:02.186943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e22de6-a5ff-4e92-91cc-8424ceaf5dd9 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:02.209972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fff6a1-a762-488a-89a3-3803072e6d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:02.209972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fff6a1-a762-488a-89a3-3803072e6d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:02.214832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fff6a1-a762-488a-89a3-3803072e6d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:02.215281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fff6a1-a762-488a-89a3-3803072e6d5a admin admin] PATH_INFO: `/v3/projects/b76cd588e38a4b3393dd2f79c2632266` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:02.219051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fff6a1-a762-488a-89a3-3803072e6d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b76cd588e38a4b3393dd2f79c2632266)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:02.224292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fff6a1-a762-488a-89a3-3803072e6d5a admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:02.334428 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e1fff6a1-a762-488a-89a3-3803072e6d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b76cd588e38a4b3393dd2f79c2632266', 'request_id': 'req-e1fff6a1-a762-488a-89a3-3803072e6d5a'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:02.354519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e1fff6a1-a762-488a-89a3-3803072e6d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b76cd588e38a4b3393dd2f79c2632266', 'request_id': 'req-e1fff6a1-a762-488a-89a3-3803072e6d5a'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:02.364834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e1fff6a1-a762-488a-89a3-3803072e6d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b76cd588e38a4b3393dd2f79c2632266', 'request_id': 'req-e1fff6a1-a762-488a-89a3-3803072e6d5a'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:02.377342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e1fff6a1-a762-488a-89a3-3803072e6d5a admin admin] The token cache is being invalidate because project b76cd588e38a4b3393dd2f79c2632266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:56:02.377763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e1fff6a1-a762-488a-89a3-3803072e6d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:02.379289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e1fff6a1-a762-488a-89a3-3803072e6d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:02.383483 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1179/2317] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:56:02 2022] DELETE /identity/v3/projects/b76cd588e38a4b3393dd2f79c2632266 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:56:02.388979 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba859f11-9d93-49c6-81ef-2ca6ad9ff65f None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:02.389457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba859f11-9d93-49c6-81ef-2ca6ad9ff65f None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:02.389717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba859f11-9d93-49c6-81ef-2ca6ad9ff65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:02.412267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-ba859f11-9d93-49c6-81ef-2ca6ad9ff65f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:02.414214 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ba859f11-9d93-49c6-81ef-2ca6ad9ff65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:02.504611 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1139/2318] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:02.505797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f5324-93de-4d3c-b941-3850394ff7d4 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:02.506069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f5324-93de-4d3c-b941-3850394ff7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:02.506340 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f5324-93de-4d3c-b941-3850394ff7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:02.543789 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-062f5324-93de-4d3c-b941-3850394ff7d4 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:02.545052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-062f5324-93de-4d3c-b941-3850394ff7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:02.568417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1180/2319] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:02.569990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba859f11-9d93-49c6-81ef-2ca6ad9ff65f None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:02.587951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba859f11-9d93-49c6-81ef-2ca6ad9ff65f None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:02.626653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132f110-a46e-4f4c-92df-eb3e89d49ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:02.627487 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132f110-a46e-4f4c-92df-eb3e89d49ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:02.627863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132f110-a46e-4f4c-92df-eb3e89d49ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:02.628204 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b132f110-a46e-4f4c-92df-eb3e89d49ed5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:02.629266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132f110-a46e-4f4c-92df-eb3e89d49ed5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:56:02.629694 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132f110-a46e-4f4c-92df-eb3e89d49ed5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:02.632410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b132f110-a46e-4f4c-92df-eb3e89d49ed5 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:02.665854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f5324-93de-4d3c-b941-3850394ff7d4 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:02.667211 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1140/2320] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:56:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:56:02.678010 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f5324-93de-4d3c-b941-3850394ff7d4 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:02.703783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55beea1-da81-4daa-b008-2986b3110679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:02.704562 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55beea1-da81-4daa-b008-2986b3110679 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:02.704947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55beea1-da81-4daa-b008-2986b3110679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:02.705329 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55beea1-da81-4daa-b008-2986b3110679 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:02.706301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55beea1-da81-4daa-b008-2986b3110679 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:02.709418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55beea1-da81-4daa-b008-2986b3110679 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:02.757029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1181/2321] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:56:02 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:56:02.758378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132f110-a46e-4f4c-92df-eb3e89d49ed5 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:02.768701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b132f110-a46e-4f4c-92df-eb3e89d49ed5 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:02.882453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd1a2f-f7ff-42e5-b6c5-85762a88fb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:02.883307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd1a2f-f7ff-42e5-b6c5-85762a88fb83 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:02.883690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd1a2f-f7ff-42e5-b6c5-85762a88fb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:02.884067 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd1a2f-f7ff-42e5-b6c5-85762a88fb83 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:02.884840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd1a2f-f7ff-42e5-b6c5-85762a88fb83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:02.890320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd1a2f-f7ff-42e5-b6c5-85762a88fb83 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:02.947747 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1141/2322] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:56:02 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:56:02.951641 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55beea1-da81-4daa-b008-2986b3110679 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:02.969900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55beea1-da81-4daa-b008-2986b3110679 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:02.976045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4198249-6986-4abf-8413-beaf35954d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:02.976797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4198249-6986-4abf-8413-beaf35954d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:02.977187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4198249-6986-4abf-8413-beaf35954d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:02.977568 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4198249-6986-4abf-8413-beaf35954d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:02.978390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4198249-6986-4abf-8413-beaf35954d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:02.980162 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4198249-6986-4abf-8413-beaf35954d98 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:02.997492 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1182/2323] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:56:03.003144 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd1a2f-f7ff-42e5-b6c5-85762a88fb83 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:03.018013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd1a2f-f7ff-42e5-b6c5-85762a88fb83 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:03.028640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4f257-5c97-4d8b-857c-95de64c20e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:03.029443 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4f257-5c97-4d8b-857c-95de64c20e56 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:03.030660 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4f257-5c97-4d8b-857c-95de64c20e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:03.031046 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4f257-5c97-4d8b-857c-95de64c20e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:03.032352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4f257-5c97-4d8b-857c-95de64c20e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:03.035884 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4f257-5c97-4d8b-857c-95de64c20e56 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:03.073145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4198249-6986-4abf-8413-beaf35954d98 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:03.081165 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1142/2324] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:56:03.089511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4198249-6986-4abf-8413-beaf35954d98 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:03.101558 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28928d-3d96-435a-bec4-728c17733717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:03.102465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28928d-3d96-435a-bec4-728c17733717 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:03.102893 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28928d-3d96-435a-bec4-728c17733717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:03.103276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb28928d-3d96-435a-bec4-728c17733717 admin admin] PATH_INFO: `/v3/projects/32d9e8ab388545eebbe8a2b7d60d5c9b/users/27136c9cafe24290bfdd4f07fea0320d/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:03.134987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28928d-3d96-435a-bec4-728c17733717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32d9e8ab388545eebbe8a2b7d60d5c9b, user_id=27136c9cafe24290bfdd4f07fea0320d, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:03.142419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb28928d-3d96-435a-bec4-728c17733717 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:03.176899 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbae80-99e8-4184-acf1-5ec361cb0256 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:03.177192 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbae80-99e8-4184-acf1-5ec361cb0256 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:03.177945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbae80-99e8-4184-acf1-5ec361cb0256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:03.180757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1183/2325] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:56:03 2022] PUT /identity/v3/projects/32d9e8ab388545eebbe8a2b7d60d5c9b/users/27136c9cafe24290bfdd4f07fea0320d/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:56:03.206394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-92dbae80-99e8-4184-acf1-5ec361cb0256 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:03.207546 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-92dbae80-99e8-4184-acf1-5ec361cb0256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:03.228922 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:03.290344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1143/2326] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:03.296713 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28928d-3d96-435a-bec4-728c17733717 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:03.298941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-fb28928d-3d96-435a-bec4-728c17733717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:03.406628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb28928d-3d96-435a-bec4-728c17733717 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:03.412546 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d96f91-61cc-4472-abba-e045d97a17bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:03.413186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d96f91-61cc-4472-abba-e045d97a17bd service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:03.413479 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d96f91-61cc-4472-abba-e045d97a17bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:03.413753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d96f91-61cc-4472-abba-e045d97a17bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:03.424644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d96f91-61cc-4472-abba-e045d97a17bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:03.426929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d96f91-61cc-4472-abba-e045d97a17bd service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:03.438055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1184/2327] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:56:10.415157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7b481-4bb4-4639-b0ae-d4a106a1fe82 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:10.416176 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7b481-4bb4-4639-b0ae-d4a106a1fe82 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:10.418920 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7b481-4bb4-4639-b0ae-d4a106a1fe82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:10.474562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-02d7b481-4bb4-4639-b0ae-d4a106a1fe82 None None] MFA Rules not processed for user `27136c9cafe24290bfdd4f07fea0320d`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:10.475780 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-02d7b481-4bb4-4639-b0ae-d4a106a1fe82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:10.588446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1144/2328] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:10.593499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d96f91-61cc-4472-abba-e045d97a17bd service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:10.604693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d96f91-61cc-4472-abba-e045d97a17bd service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:10.609821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78154455-9a46-4aeb-8fd6-83edb56e9148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:10.610617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78154455-9a46-4aeb-8fd6-83edb56e9148 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:10.611004 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78154455-9a46-4aeb-8fd6-83edb56e9148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:10.611384 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78154455-9a46-4aeb-8fd6-83edb56e9148 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:10.612165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78154455-9a46-4aeb-8fd6-83edb56e9148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:10.614788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78154455-9a46-4aeb-8fd6-83edb56e9148 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:10.631888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:10.646732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1185/2329] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:56:10 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:56:10.650938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7b481-4bb4-4639-b0ae-d4a106a1fe82 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:10.663674 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7b481-4bb4-4639-b0ae-d4a106a1fe82 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:10.723532 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260dcd7a-9952-408b-bc1c-84a152a6fd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:10.724194 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dcd7a-9952-408b-bc1c-84a152a6fd80 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:10.724472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dcd7a-9952-408b-bc1c-84a152a6fd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:10.724738 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260dcd7a-9952-408b-bc1c-84a152a6fd80 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:10.725491 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260dcd7a-9952-408b-bc1c-84a152a6fd80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:10.727479 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260dcd7a-9952-408b-bc1c-84a152a6fd80 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:10.750559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1145/2330] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:56:10 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:56:10.754710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78154455-9a46-4aeb-8fd6-83edb56e9148 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:10.765492 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78154455-9a46-4aeb-8fd6-83edb56e9148 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:10.770397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2e7dc-1de2-4789-b7da-88a079199b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:10.771054 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e7dc-1de2-4789-b7da-88a079199b54 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:10.771331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e7dc-1de2-4789-b7da-88a079199b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:10.771601 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e7dc-1de2-4789-b7da-88a079199b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:10.772297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2e7dc-1de2-4789-b7da-88a079199b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:10.773889 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2e7dc-1de2-4789-b7da-88a079199b54 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:10.785599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1186/2331] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:56:10.789886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260dcd7a-9952-408b-bc1c-84a152a6fd80 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:10.810423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260dcd7a-9952-408b-bc1c-84a152a6fd80 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:10.820749 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d6c54-3a15-4483-b4c0-7c9dcfbeef83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:10.821257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d6c54-3a15-4483-b4c0-7c9dcfbeef83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:10.821538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d6c54-3a15-4483-b4c0-7c9dcfbeef83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:10.821814 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0d6c54-3a15-4483-b4c0-7c9dcfbeef83 admin admin] PATH_INFO: `/v3/projects/f85a52d1aa3648108739242d9c3f824b/users/99a660a9790f438fa0ee4d376f7e6707/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:10.838384 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d6c54-3a15-4483-b4c0-7c9dcfbeef83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85a52d1aa3648108739242d9c3f824b, user_id=99a660a9790f438fa0ee4d376f7e6707, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:10.841137 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0d6c54-3a15-4483-b4c0-7c9dcfbeef83 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:10.867514 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1146/2332] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:56:10 2022] PUT /identity/v3/projects/f85a52d1aa3648108739242d9c3f824b/users/99a660a9790f438fa0ee4d376f7e6707/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:56:10.869440 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2e7dc-1de2-4789-b7da-88a079199b54 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:10.885685 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2e7dc-1de2-4789-b7da-88a079199b54 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:10.964654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91cca6-f041-4d94-941c-c50e0b6831b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:10.965441 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91cca6-f041-4d94-941c-c50e0b6831b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:10.965642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91cca6-f041-4d94-941c-c50e0b6831b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:10.965915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91cca6-f041-4d94-941c-c50e0b6831b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:10.966672 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91cca6-f041-4d94-941c-c50e0b6831b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:10.968294 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91cca6-f041-4d94-941c-c50e0b6831b3 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:10.983813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1187/2333] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:56:10.985176 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d6c54-3a15-4483-b4c0-7c9dcfbeef83 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:10.997372 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0d6c54-3a15-4483-b4c0-7c9dcfbeef83 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:11.003470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b255415-122d-431a-a802-87166a235e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:11.004554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b255415-122d-431a-a802-87166a235e9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:11.004554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b255415-122d-431a-a802-87166a235e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:11.004837 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b255415-122d-431a-a802-87166a235e9d admin admin] PATH_INFO: `/v3/projects/f85a52d1aa3648108739242d9c3f824b/users/99a660a9790f438fa0ee4d376f7e6707/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:11.007007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b255415-122d-431a-a802-87166a235e9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f85a52d1aa3648108739242d9c3f824b, user_id=99a660a9790f438fa0ee4d376f7e6707, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:11.009630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b255415-122d-431a-a802-87166a235e9d admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:11.017953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:11.026559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1147/2334] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:56:10 2022] PUT /identity/v3/projects/f85a52d1aa3648108739242d9c3f824b/users/99a660a9790f438fa0ee4d376f7e6707/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:56:11.637135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91cca6-f041-4d94-941c-c50e0b6831b3 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:11.639182 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1f91cca6-f041-4d94-941c-c50e0b6831b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:11.656344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91cca6-f041-4d94-941c-c50e0b6831b3 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:11.731113 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfdc2d-3c90-4fe2-97fe-d652045abca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:11.731693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfdc2d-3c90-4fe2-97fe-d652045abca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:11.731974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfdc2d-3c90-4fe2-97fe-d652045abca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:11.732248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dfdc2d-3c90-4fe2-97fe-d652045abca6 admin admin] PATH_INFO: `/v3/users/40087e790eaa48a29f4fc2db0a35963a` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:11.733439 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfdc2d-3c90-4fe2-97fe-d652045abca6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40087e790eaa48a29f4fc2db0a35963a)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:11.735377 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dfdc2d-3c90-4fe2-97fe-d652045abca6 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:11.790778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-28dfdc2d-3c90-4fe2-97fe-d652045abca6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40087e790eaa48a29f4fc2db0a35963a'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:11.797696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-28dfdc2d-3c90-4fe2-97fe-d652045abca6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40087e790eaa48a29f4fc2db0a35963a'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:11.811616 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-28dfdc2d-3c90-4fe2-97fe-d652045abca6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40087e790eaa48a29f4fc2db0a35963a'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:11.812402 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-28dfdc2d-3c90-4fe2-97fe-d652045abca6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40087e790eaa48a29f4fc2db0a35963a'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:11.813057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-28dfdc2d-3c90-4fe2-97fe-d652045abca6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40087e790eaa48a29f4fc2db0a35963a'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:11.824810 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:11.830320 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1188/2335] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:56:11 2022] DELETE /identity/v3/users/40087e790eaa48a29f4fc2db0a35963a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:56:11.834867 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3598ee-98bd-44ed-8785-83c8b5e950ac None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:11.835380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3598ee-98bd-44ed-8785-83c8b5e950ac None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:11.835496 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3598ee-98bd-44ed-8785-83c8b5e950ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:11.855459 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-5c3598ee-98bd-44ed-8785-83c8b5e950ac None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:11.856680 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-5c3598ee-98bd-44ed-8785-83c8b5e950ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:11.926699 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1148/2336] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:11.933277 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfdc2d-3c90-4fe2-97fe-d652045abca6 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:11.935037 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-28dfdc2d-3c90-4fe2-97fe-d652045abca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:12.007304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dfdc2d-3c90-4fe2-97fe-d652045abca6 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:12.011737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb076d-f55a-4310-a4e7-801a69906d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:12.012521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb076d-f55a-4310-a4e7-801a69906d61 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:12.012736 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb076d-f55a-4310-a4e7-801a69906d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:12.013160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb076d-f55a-4310-a4e7-801a69906d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:12.035549 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb076d-f55a-4310-a4e7-801a69906d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:12.037223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb076d-f55a-4310-a4e7-801a69906d61 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:12.052784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1189/2337] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:56:12.398044 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3598ee-98bd-44ed-8785-83c8b5e950ac None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:12.399519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-5c3598ee-98bd-44ed-8785-83c8b5e950ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:12.420485 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3598ee-98bd-44ed-8785-83c8b5e950ac None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:12.439702 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7059c-0896-46af-831d-30b5278bc816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:12.440803 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7059c-0896-46af-831d-30b5278bc816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:12.440803 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7059c-0896-46af-831d-30b5278bc816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:12.441004 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7059c-0896-46af-831d-30b5278bc816 admin admin] PATH_INFO: `/v3/projects/519af30a388145bab2fea77c2b23d191` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:12.442119 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7059c-0896-46af-831d-30b5278bc816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=519af30a388145bab2fea77c2b23d191)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:12.444686 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7059c-0896-46af-831d-30b5278bc816 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:12.497982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e5a7059c-0896-46af-831d-30b5278bc816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '519af30a388145bab2fea77c2b23d191', 'request_id': 'req-e5a7059c-0896-46af-831d-30b5278bc816'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:12.511663 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e5a7059c-0896-46af-831d-30b5278bc816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '519af30a388145bab2fea77c2b23d191', 'request_id': 'req-e5a7059c-0896-46af-831d-30b5278bc816'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:12.522957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e5a7059c-0896-46af-831d-30b5278bc816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '519af30a388145bab2fea77c2b23d191', 'request_id': 'req-e5a7059c-0896-46af-831d-30b5278bc816'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:12.538180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e5a7059c-0896-46af-831d-30b5278bc816 admin admin] The token cache is being invalidate because project 519af30a388145bab2fea77c2b23d191 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:56:12.538581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e5a7059c-0896-46af-831d-30b5278bc816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:12.540542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e5a7059c-0896-46af-831d-30b5278bc816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:12.546956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1149/2338] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:56:12 2022] DELETE /identity/v3/projects/519af30a388145bab2fea77c2b23d191 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:56:12.549530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0ad5b-2141-4637-ac8c-28a48a9878b8 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:12.549956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0ad5b-2141-4637-ac8c-28a48a9878b8 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:12.550364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0ad5b-2141-4637-ac8c-28a48a9878b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:12.578649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-12f0ad5b-2141-4637-ac8c-28a48a9878b8 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:12.581742 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-12f0ad5b-2141-4637-ac8c-28a48a9878b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:12.681835 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1190/2339] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:12.682509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc57d05-70c0-4a70-bd88-018689ce427a None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:12.682656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc57d05-70c0-4a70-bd88-018689ce427a None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:12.682932 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc57d05-70c0-4a70-bd88-018689ce427a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:12.714536 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-fbc57d05-70c0-4a70-bd88-018689ce427a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:12.715979 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-fbc57d05-70c0-4a70-bd88-018689ce427a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:12.728290 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1150/2340] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:56:12.732812 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0ad5b-2141-4637-ac8c-28a48a9878b8 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:12.754143 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0ad5b-2141-4637-ac8c-28a48a9878b8 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:12.764625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526272d-9cdd-4675-a236-116d706faa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:12.766606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526272d-9cdd-4675-a236-116d706faa80 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:12.767010 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526272d-9cdd-4675-a236-116d706faa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:12.769855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526272d-9cdd-4675-a236-116d706faa80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:12.770892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526272d-9cdd-4675-a236-116d706faa80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:56:12.771356 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526272d-9cdd-4675-a236-116d706faa80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:12.775014 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526272d-9cdd-4675-a236-116d706faa80 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:12.808783 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc57d05-70c0-4a70-bd88-018689ce427a None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:12.818460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1191/2341] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:56:12.821403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc57d05-70c0-4a70-bd88-018689ce427a None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:12.832062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6b233-cc3b-43bc-a67d-3e24e05dc929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:12.834046 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6b233-cc3b-43bc-a67d-3e24e05dc929 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:12.835619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6b233-cc3b-43bc-a67d-3e24e05dc929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:12.836166 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6b233-cc3b-43bc-a67d-3e24e05dc929 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:12.838795 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea6b233-cc3b-43bc-a67d-3e24e05dc929 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:12.841633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea6b233-cc3b-43bc-a67d-3e24e05dc929 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:12.890250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1151/2342] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:56:12 2022] POST /identity/v3/projects => generated 371 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:56:12.895100 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526272d-9cdd-4675-a236-116d706faa80 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:12.907050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526272d-9cdd-4675-a236-116d706faa80 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:13.063816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68194e16-b786-4b3a-bd78-09d62742f086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:13.064638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68194e16-b786-4b3a-bd78-09d62742f086 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:13.064930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68194e16-b786-4b3a-bd78-09d62742f086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:13.066828 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68194e16-b786-4b3a-bd78-09d62742f086 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:13.067544 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68194e16-b786-4b3a-bd78-09d62742f086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:13.069722 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68194e16-b786-4b3a-bd78-09d62742f086 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:13.125790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1192/2343] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:56:12 2022] POST /identity/v3/users => generated 286 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:56:13.130618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6b233-cc3b-43bc-a67d-3e24e05dc929 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:13.144612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6b233-cc3b-43bc-a67d-3e24e05dc929 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:13.149976 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde0e84-882c-4066-adec-befcce412695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:13.150842 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde0e84-882c-4066-adec-befcce412695 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:13.151238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde0e84-882c-4066-adec-befcce412695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:13.151618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde0e84-882c-4066-adec-befcce412695 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:13.152437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde0e84-882c-4066-adec-befcce412695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:13.154250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde0e84-882c-4066-adec-befcce412695 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:13.169550 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1152/2344] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:56:13.173795 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68194e16-b786-4b3a-bd78-09d62742f086 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:13.193932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68194e16-b786-4b3a-bd78-09d62742f086 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:13.203055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a67aa-ca42-41af-a2a5-aa9e815f91d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:13.204351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a67aa-ca42-41af-a2a5-aa9e815f91d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:13.205033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a67aa-ca42-41af-a2a5-aa9e815f91d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:13.205409 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a67aa-ca42-41af-a2a5-aa9e815f91d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:13.206738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a67aa-ca42-41af-a2a5-aa9e815f91d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:13.209592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a67aa-ca42-41af-a2a5-aa9e815f91d5 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:13.235878 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1193/2345] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:56:13.239290 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde0e84-882c-4066-adec-befcce412695 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:13.253779 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde0e84-882c-4066-adec-befcce412695 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:13.259880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2244f-0aa8-434c-9f53-8cb1e4b984d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:13.261565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2244f-0aa8-434c-9f53-8cb1e4b984d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:13.262067 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2244f-0aa8-434c-9f53-8cb1e4b984d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:13.262535 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2244f-0aa8-434c-9f53-8cb1e4b984d0 admin admin] PATH_INFO: `/v3/projects/02d97e03450e461e97179390f4c022b0/users/b83cdb23e81c4763bbc34a6a74720b06/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:13.287416 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2244f-0aa8-434c-9f53-8cb1e4b984d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02d97e03450e461e97179390f4c022b0, user_id=b83cdb23e81c4763bbc34a6a74720b06, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:13.290510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2244f-0aa8-434c-9f53-8cb1e4b984d0 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:13.309968 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1153/2346] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:56:13 2022] PUT /identity/v3/projects/02d97e03450e461e97179390f4c022b0/users/b83cdb23e81c4763bbc34a6a74720b06/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:56:13.320154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae0efd-406d-4608-8805-7a9b8db7cd90 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:13.320467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae0efd-406d-4608-8805-7a9b8db7cd90 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:13.320764 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae0efd-406d-4608-8805-7a9b8db7cd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:13.342983 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:13.358053 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1cae0efd-406d-4608-8805-7a9b8db7cd90 None None] MFA Rules not processed for user `b83cdb23e81c4763bbc34a6a74720b06`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:13.359433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1cae0efd-406d-4608-8805-7a9b8db7cd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:13.461837 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1194/2347] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:13.470835 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2244f-0aa8-434c-9f53-8cb1e4b984d0 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:13.473250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8ec2244f-0aa8-434c-9f53-8cb1e4b984d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:13.499261 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:13.544939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2244f-0aa8-434c-9f53-8cb1e4b984d0 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:13.548738 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf806570-193e-41f9-8239-3fe3b7432d06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:13.549618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf806570-193e-41f9-8239-3fe3b7432d06 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:13.549923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf806570-193e-41f9-8239-3fe3b7432d06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:13.550524 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf806570-193e-41f9-8239-3fe3b7432d06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:13.559620 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf806570-193e-41f9-8239-3fe3b7432d06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:13.561150 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf806570-193e-41f9-8239-3fe3b7432d06 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:13.583862 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1154/2348] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:56:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:56:13.598038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae0efd-406d-4608-8805-7a9b8db7cd90 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:13.686020 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae0efd-406d-4608-8805-7a9b8db7cd90 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:13.692142 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640c2dd-a839-4181-8c5a-5e0ab4e2b043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:13.692748 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640c2dd-a839-4181-8c5a-5e0ab4e2b043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:13.693035 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640c2dd-a839-4181-8c5a-5e0ab4e2b043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:13.693311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640c2dd-a839-4181-8c5a-5e0ab4e2b043 admin admin] PATH_INFO: `/v3/users/b83cdb23e81c4763bbc34a6a74720b06` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:13.694981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640c2dd-a839-4181-8c5a-5e0ab4e2b043 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b83cdb23e81c4763bbc34a6a74720b06)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:13.696686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f640c2dd-a839-4181-8c5a-5e0ab4e2b043 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:13.755297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f640c2dd-a839-4181-8c5a-5e0ab4e2b043 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b83cdb23e81c4763bbc34a6a74720b06'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:13.765683 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f640c2dd-a839-4181-8c5a-5e0ab4e2b043 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b83cdb23e81c4763bbc34a6a74720b06'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:13.784150 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f640c2dd-a839-4181-8c5a-5e0ab4e2b043 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b83cdb23e81c4763bbc34a6a74720b06'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:13.784995 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f640c2dd-a839-4181-8c5a-5e0ab4e2b043 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b83cdb23e81c4763bbc34a6a74720b06'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:13.785665 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f640c2dd-a839-4181-8c5a-5e0ab4e2b043 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b83cdb23e81c4763bbc34a6a74720b06'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:13.810765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1195/2349] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:56:13 2022] DELETE /identity/v3/users/b83cdb23e81c4763bbc34a6a74720b06 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:56:13.811816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039bc4a-bce2-4112-98c8-8521678d8bfe None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:13.812112 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039bc4a-bce2-4112-98c8-8521678d8bfe None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:13.813024 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039bc4a-bce2-4112-98c8-8521678d8bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:13.838629 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-d039bc4a-bce2-4112-98c8-8521678d8bfe None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:13.840021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d039bc4a-bce2-4112-98c8-8521678d8bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:13.920808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1155/2350] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:13.928769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640c2dd-a839-4181-8c5a-5e0ab4e2b043 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:13.930396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f640c2dd-a839-4181-8c5a-5e0ab4e2b043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:13.967229 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:14.017714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f640c2dd-a839-4181-8c5a-5e0ab4e2b043 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:14.025978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b0657-d9e0-4414-83dc-c0b809b8cf26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:14.027312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b0657-d9e0-4414-83dc-c0b809b8cf26 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:14.027312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b0657-d9e0-4414-83dc-c0b809b8cf26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:14.028012 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b0657-d9e0-4414-83dc-c0b809b8cf26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:14.039981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b0657-d9e0-4414-83dc-c0b809b8cf26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:14.041724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b0657-d9e0-4414-83dc-c0b809b8cf26 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:14.064081 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1196/2351] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:56:14.122526 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039bc4a-bce2-4112-98c8-8521678d8bfe None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:14.125006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d039bc4a-bce2-4112-98c8-8521678d8bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:14.330111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039bc4a-bce2-4112-98c8-8521678d8bfe None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:14.351444 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca202d4-1446-46d9-97ba-dd13e64a350d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5094263ac471481db48783b19192e01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:14.352689 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca202d4-1446-46d9-97ba-dd13e64a350d service placement] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:14.352988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca202d4-1446-46d9-97ba-dd13e64a350d service placement] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:14.354763 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca202d4-1446-46d9-97ba-dd13e64a350d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:14.360634 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6ca202d4-1446-46d9-97ba-dd13e64a350d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:14.386743 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca202d4-1446-46d9-97ba-dd13e64a350d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:14.388395 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca202d4-1446-46d9-97ba-dd13e64a350d service placement] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:14.417022 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1156/2352] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:56:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:56:14.588952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b0657-d9e0-4414-83dc-c0b809b8cf26 service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:14.590683 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-348b0657-d9e0-4414-83dc-c0b809b8cf26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:14.613268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b0657-d9e0-4414-83dc-c0b809b8cf26 service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:14.628547 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7730b-6534-4146-bc16-40658461da2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:14.630108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7730b-6534-4146-bc16-40658461da2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:14.630649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7730b-6534-4146-bc16-40658461da2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:14.631192 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7730b-6534-4146-bc16-40658461da2f admin admin] PATH_INFO: `/v3/projects/02d97e03450e461e97179390f4c022b0` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:14.632801 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7730b-6534-4146-bc16-40658461da2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=02d97e03450e461e97179390f4c022b0)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:14.637462 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7730b-6534-4146-bc16-40658461da2f admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:14.740424 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1ab7730b-6534-4146-bc16-40658461da2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02d97e03450e461e97179390f4c022b0', 'request_id': 'req-1ab7730b-6534-4146-bc16-40658461da2f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:14.755022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1ab7730b-6534-4146-bc16-40658461da2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02d97e03450e461e97179390f4c022b0', 'request_id': 'req-1ab7730b-6534-4146-bc16-40658461da2f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:14.764458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1ab7730b-6534-4146-bc16-40658461da2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02d97e03450e461e97179390f4c022b0', 'request_id': 'req-1ab7730b-6534-4146-bc16-40658461da2f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:14.778077 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1ab7730b-6534-4146-bc16-40658461da2f admin admin] The token cache is being invalidate because project 02d97e03450e461e97179390f4c022b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:56:14.778546 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1ab7730b-6534-4146-bc16-40658461da2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:14.779315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1ab7730b-6534-4146-bc16-40658461da2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:14.784412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1197/2353] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:56:14 2022] DELETE /identity/v3/projects/02d97e03450e461e97179390f4c022b0 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:56:14.786963 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcc1c0-eee4-4e47-b61a-d1f91954db7d None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:14.787445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcc1c0-eee4-4e47-b61a-d1f91954db7d None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:14.787986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fcc1c0-eee4-4e47-b61a-d1f91954db7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:14.814006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-e4fcc1c0-eee4-4e47-b61a-d1f91954db7d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:14.815568 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e4fcc1c0-eee4-4e47-b61a-d1f91954db7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:14.901711 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:14.947635 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1157/2354] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:14.949676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e7fbd-d240-4945-882e-846e0cfdabbc None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:14.950153 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e7fbd-d240-4945-882e-846e0cfdabbc None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:14.950553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24e7fbd-d240-4945-882e-846e0cfdabbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:14.999425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-a24e7fbd-d240-4945-882e-846e0cfdabbc None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:15.004577 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a24e7fbd-d240-4945-882e-846e0cfdabbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:15.028065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1198/2355] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:15.030693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcc1c0-eee4-4e47-b61a-d1f91954db7d None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:15.048199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fcc1c0-eee4-4e47-b61a-d1f91954db7d None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:15.077836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b718ce2-5916-456b-8aab-9aecba09a6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:15.078748 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b718ce2-5916-456b-8aab-9aecba09a6da admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:15.079037 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b718ce2-5916-456b-8aab-9aecba09a6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:15.079314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b718ce2-5916-456b-8aab-9aecba09a6da admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:15.080106 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b718ce2-5916-456b-8aab-9aecba09a6da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:56:15.080416 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b718ce2-5916-456b-8aab-9aecba09a6da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:15.083334 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b718ce2-5916-456b-8aab-9aecba09a6da admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:15.110157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1158/2356] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:56:15.114150 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e7fbd-d240-4945-882e-846e0cfdabbc None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:15.141187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24e7fbd-d240-4945-882e-846e0cfdabbc None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:15.150232 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761bcf1-ce2e-48be-87dc-ffde0783a64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:15.162067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761bcf1-ce2e-48be-87dc-ffde0783a64d admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:15.164756 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761bcf1-ce2e-48be-87dc-ffde0783a64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:15.165730 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761bcf1-ce2e-48be-87dc-ffde0783a64d admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:15.182573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761bcf1-ce2e-48be-87dc-ffde0783a64d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:15.192437 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761bcf1-ce2e-48be-87dc-ffde0783a64d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:15.266206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1199/2357] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:56:15 2022] POST /identity/v3/projects => generated 373 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:56:15.269656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b718ce2-5916-456b-8aab-9aecba09a6da admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:15.285203 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b718ce2-5916-456b-8aab-9aecba09a6da admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:15.432105 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9383048-7e78-4b57-8822-7b22c2449119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:15.433219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9383048-7e78-4b57-8822-7b22c2449119 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:15.433503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9383048-7e78-4b57-8822-7b22c2449119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:15.433812 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9383048-7e78-4b57-8822-7b22c2449119 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:15.434712 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9383048-7e78-4b57-8822-7b22c2449119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:15.438436 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9383048-7e78-4b57-8822-7b22c2449119 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:15.483971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1159/2358] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:56:15 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:56:15.485207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761bcf1-ce2e-48be-87dc-ffde0783a64d admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:15.501144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761bcf1-ce2e-48be-87dc-ffde0783a64d admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:15.512310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76fdf4-51cd-48cd-a498-0a85bae47008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:15.513222 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76fdf4-51cd-48cd-a498-0a85bae47008 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:15.513497 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76fdf4-51cd-48cd-a498-0a85bae47008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:15.513786 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76fdf4-51cd-48cd-a498-0a85bae47008 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:15.514646 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76fdf4-51cd-48cd-a498-0a85bae47008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:15.517618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76fdf4-51cd-48cd-a498-0a85bae47008 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:15.535641 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1200/2359] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:56:15.536868 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9383048-7e78-4b57-8822-7b22c2449119 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:15.551950 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9383048-7e78-4b57-8822-7b22c2449119 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:15.559056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150db4f-97fd-4728-aa99-df1cd282d869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:15.559690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150db4f-97fd-4728-aa99-df1cd282d869 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:15.559973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150db4f-97fd-4728-aa99-df1cd282d869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:15.560302 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150db4f-97fd-4728-aa99-df1cd282d869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:15.561010 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150db4f-97fd-4728-aa99-df1cd282d869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:15.563166 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150db4f-97fd-4728-aa99-df1cd282d869 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:15.580108 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1160/2360] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:56:15.581980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76fdf4-51cd-48cd-a498-0a85bae47008 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:15.593070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76fdf4-51cd-48cd-a498-0a85bae47008 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:15.601689 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c5e6e-4e6e-4ef1-b78d-f706c485918f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:15.602688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c5e6e-4e6e-4ef1-b78d-f706c485918f admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:15.603160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c5e6e-4e6e-4ef1-b78d-f706c485918f admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:15.603643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c5e6e-4e6e-4ef1-b78d-f706c485918f admin admin] PATH_INFO: `/v3/projects/39cef51c667e481c8f385d547d6a57da/users/381f1faddc1b4905b92ebba918213df4/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:15.625122 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c5e6e-4e6e-4ef1-b78d-f706c485918f admin admin] RBAC: Authorizing `identity:create_grant(project_id=39cef51c667e481c8f385d547d6a57da, user_id=381f1faddc1b4905b92ebba918213df4, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:15.630429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c5e6e-4e6e-4ef1-b78d-f706c485918f admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:15.648006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1201/2361] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:56:15 2022] PUT /identity/v3/projects/39cef51c667e481c8f385d547d6a57da/users/381f1faddc1b4905b92ebba918213df4/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:56:15.719127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea608640-b09a-4b30-b3c5-4cde91664fbc None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:15.721188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea608640-b09a-4b30-b3c5-4cde91664fbc None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:15.721188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea608640-b09a-4b30-b3c5-4cde91664fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:15.756838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-ea608640-b09a-4b30-b3c5-4cde91664fbc None None] MFA Rules not processed for user `381f1faddc1b4905b92ebba918213df4`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:15.758187 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ea608640-b09a-4b30-b3c5-4cde91664fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:15.838759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1161/2362] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:56:15.850984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c5e6e-4e6e-4ef1-b78d-f706c485918f admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:15.853481 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8a9c5e6e-4e6e-4ef1-b78d-f706c485918f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:15.917273 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:15.937968 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c5e6e-4e6e-4ef1-b78d-f706c485918f admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:15.943426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe707d8-5309-43f2-9eaf-2c28014940f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:15.944449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe707d8-5309-43f2-9eaf-2c28014940f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:15.944810 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe707d8-5309-43f2-9eaf-2c28014940f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:15.945167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe707d8-5309-43f2-9eaf-2c28014940f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:15.957281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe707d8-5309-43f2-9eaf-2c28014940f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:15.959235 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe707d8-5309-43f2-9eaf-2c28014940f9 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:15.970117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1202/2363] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:56:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:56:16.951876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb661734-a185-4c03-a86b-8445799ad3b0 None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:16.952414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb661734-a185-4c03-a86b-8445799ad3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:16.952558 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb661734-a185-4c03-a86b-8445799ad3b0 None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:16.954381 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1162/2364] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 3 12:56:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:56:16.969298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da8e05-a6d8-4f70-9240-80eefb1c2506 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:16.969896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da8e05-a6d8-4f70-9240-80eefb1c2506 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:16.970142 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da8e05-a6d8-4f70-9240-80eefb1c2506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:17.009060 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-43da8e05-a6d8-4f70-9240-80eefb1c2506 None None] MFA Rules not processed for user `7b634f567dca42dc95df175a4f36c1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:17.010436 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-43da8e05-a6d8-4f70-9240-80eefb1c2506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:17.157106 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd84bc7-238b-4d48-8a52-56539d5392a3 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:17.157460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd84bc7-238b-4d48-8a52-56539d5392a3 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:17.158026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd84bc7-238b-4d48-8a52-56539d5392a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:17.169692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1203/2365] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 3 12:56:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:56:17.192791 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-acd84bc7-238b-4d48-8a52-56539d5392a3 None None] MFA Rules not processed for user `99a660a9790f438fa0ee4d376f7e6707`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:17.194912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-acd84bc7-238b-4d48-8a52-56539d5392a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:17.204013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da8e05-a6d8-4f70-9240-80eefb1c2506 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:17.210640 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-43da8e05-a6d8-4f70-9240-80eefb1c2506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:17.259020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:17.292662 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da8e05-a6d8-4f70-9240-80eefb1c2506 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:17.296347 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788ad32-7563-4db3-8b7a-6a6e35fe6f87 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6066e48aa644a47a889e09cfe204858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:17.297229 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788ad32-7563-4db3-8b7a-6a6e35fe6f87 service glance] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:17.297570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788ad32-7563-4db3-8b7a-6a6e35fe6f87 service glance] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:17.297908 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788ad32-7563-4db3-8b7a-6a6e35fe6f87 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:17.301679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1163/2366] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:56:17.327782 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788ad32-7563-4db3-8b7a-6a6e35fe6f87 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:17.329711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788ad32-7563-4db3-8b7a-6a6e35fe6f87 service glance] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:17.343356 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1204/2367] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:56:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:56:17.380110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd84bc7-238b-4d48-8a52-56539d5392a3 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:17.381597 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-acd84bc7-238b-4d48-8a52-56539d5392a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:17.431847 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd84bc7-238b-4d48-8a52-56539d5392a3 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:17.440186 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d193b-066a-4fc7-ba05-61a0e3d48281 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6066e48aa644a47a889e09cfe204858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:17.440943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d193b-066a-4fc7-ba05-61a0e3d48281 service glance] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:17.441229 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d193b-066a-4fc7-ba05-61a0e3d48281 service glance] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:17.441553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d193b-066a-4fc7-ba05-61a0e3d48281 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:17.443724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f52d193b-066a-4fc7-ba05-61a0e3d48281 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:17.458849 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e193fdf-89d1-4dc9-a02f-70a82fbfc4f6 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:17.459132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e193fdf-89d1-4dc9-a02f-70a82fbfc4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:17.459411 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e193fdf-89d1-4dc9-a02f-70a82fbfc4f6 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:17.460750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1205/2368] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 3 12:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:56:17.468430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93830d56-f11d-4e50-bb5c-030b00fe9e41 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:17.468634 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93830d56-f11d-4e50-bb5c-030b00fe9e41 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:17.470194 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93830d56-f11d-4e50-bb5c-030b00fe9e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:17.494277 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-93830d56-f11d-4e50-bb5c-030b00fe9e41 None None] MFA Rules not processed for user `52ff458fd7f74b99a745edbcf13a114b`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:17.495654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-93830d56-f11d-4e50-bb5c-030b00fe9e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:17.556891 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d193b-066a-4fc7-ba05-61a0e3d48281 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:17.560148 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d193b-066a-4fc7-ba05-61a0e3d48281 service glance] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:17.576107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1206/2369] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 3 12:56:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:56:17.578414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376de95-a213-4158-83bf-79834387e827 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:17.578797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376de95-a213-4158-83bf-79834387e827 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:17.579097 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376de95-a213-4158-83bf-79834387e827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:17.584950 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1164/2370] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:56:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:56:17.591767 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d193b-066a-4fc7-ba05-61a0e3d48281 service glance] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:17.595272 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f52d193b-066a-4fc7-ba05-61a0e3d48281 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:17.601009 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-e376de95-a213-4158-83bf-79834387e827 None None] MFA Rules not processed for user `381f1faddc1b4905b92ebba918213df4`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:17.602205 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e376de95-a213-4158-83bf-79834387e827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:17.610843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1207/2371] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:56:17.616301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d193b-066a-4fc7-ba05-61a0e3d48281 service glance] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:17.621274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c8586-4d8c-4a81-8a59-f4427add0b6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:17.621948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c8586-4d8c-4a81-8a59-f4427add0b6c service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:17.622228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c8586-4d8c-4a81-8a59-f4427add0b6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:17.622519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c8586-4d8c-4a81-8a59-f4427add0b6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:17.631343 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c8586-4d8c-4a81-8a59-f4427add0b6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:17.632780 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c8586-4d8c-4a81-8a59-f4427add0b6c service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:17.665886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1165/2372] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:56:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:56:17.736563 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367149f-29c2-4c2e-9069-226a6c4ec7eb None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:17.736846 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367149f-29c2-4c2e-9069-226a6c4ec7eb None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:17.737102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e367149f-29c2-4c2e-9069-226a6c4ec7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:17.757362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-e367149f-29c2-4c2e-9069-226a6c4ec7eb None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:17.758632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e367149f-29c2-4c2e-9069-226a6c4ec7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:17.772004 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25e01f-a660-4a46-8af6-fe7992855e3b None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:17.772295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25e01f-a660-4a46-8af6-fe7992855e3b None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:17.772578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25e01f-a660-4a46-8af6-fe7992855e3b None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:17.774026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:17.775275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1166/2373] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 3 12:56:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 03 12:56:17.780944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68791196-45c9-4772-a6f6-f46951b166b8 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:17.781353 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68791196-45c9-4772-a6f6-f46951b166b8 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:17.781716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68791196-45c9-4772-a6f6-f46951b166b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:17.803622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-68791196-45c9-4772-a6f6-f46951b166b8 None None] MFA Rules not processed for user `7b634f567dca42dc95df175a4f36c1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:17.805486 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-68791196-45c9-4772-a6f6-f46951b166b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:17.823510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1167/2374] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 3 12:56:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:56:17.830326 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68791196-45c9-4772-a6f6-f46951b166b8 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:17.832712 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-68791196-45c9-4772-a6f6-f46951b166b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:17.846237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68791196-45c9-4772-a6f6-f46951b166b8 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:17.849022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1208/2375] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 3 12:56:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:17.852961 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847a2f8-5cc0-446e-9c7c-542891ab0bdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:17.853908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847a2f8-5cc0-446e-9c7c-542891ab0bdd service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:17.854458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847a2f8-5cc0-446e-9c7c-542891ab0bdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:17.854940 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847a2f8-5cc0-446e-9c7c-542891ab0bdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:17.871277 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367149f-29c2-4c2e-9069-226a6c4ec7eb None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:17.873194 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847a2f8-5cc0-446e-9c7c-542891ab0bdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:17.875134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4847a2f8-5cc0-446e-9c7c-542891ab0bdd service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:17.881060 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e367149f-29c2-4c2e-9069-226a6c4ec7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:17.893355 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1168/2376] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:56:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:56:18.032316 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e367149f-29c2-4c2e-9069-226a6c4ec7eb None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:18.039280 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cae77-41fd-410b-8870-b36a80ae46d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:18.039911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cae77-41fd-410b-8870-b36a80ae46d5 service swift] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:18.040190 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cae77-41fd-410b-8870-b36a80ae46d5 service swift] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:18.040452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791cae77-41fd-410b-8870-b36a80ae46d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:18.058216 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791cae77-41fd-410b-8870-b36a80ae46d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:18.059853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791cae77-41fd-410b-8870-b36a80ae46d5 service swift] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:18.069884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1209/2377] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:56:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 03 12:56:21.830776 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4cdc2-705c-49e4-9d20-d90da3afeb72 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:21.832355 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4cdc2-705c-49e4-9d20-d90da3afeb72 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:21.832857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4cdc2-705c-49e4-9d20-d90da3afeb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:21.872603 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-0cc4cdc2-705c-49e4-9d20-d90da3afeb72 None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:21.874367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-0cc4cdc2-705c-49e4-9d20-d90da3afeb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:21.892989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1169/2378] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 3 12:56:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:56:21.945499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cae77-41fd-410b-8870-b36a80ae46d5 service swift] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:21.982791 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791cae77-41fd-410b-8870-b36a80ae46d5 service swift] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:21.993741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7f107-3527-4435-b646-8df94897163a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:21.994866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7f107-3527-4435-b646-8df94897163a service swift] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:21.995311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7f107-3527-4435-b646-8df94897163a service swift] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:21.995741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7f107-3527-4435-b646-8df94897163a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:22.008230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7f107-3527-4435-b646-8df94897163a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:22.010219 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7f107-3527-4435-b646-8df94897163a service swift] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:22.050565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1210/2379] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:56:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 12:56:28.701770 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c62589-ad27-4e40-961a-fb8eaffe1aaf None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:28.702371 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c62589-ad27-4e40-961a-fb8eaffe1aaf None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:28.702371 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c62589-ad27-4e40-961a-fb8eaffe1aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:28.723049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-83c62589-ad27-4e40-961a-fb8eaffe1aaf None None] MFA Rules not processed for user `27136c9cafe24290bfdd4f07fea0320d`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:28.724320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-83c62589-ad27-4e40-961a-fb8eaffe1aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:28.732068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1170/2380] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:32.020094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=54 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:36.090456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610710e-2380-44a4-9907-3ca6869821c1 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:36.091105 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610710e-2380-44a4-9907-3ca6869821c1 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:36.091105 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610710e-2380-44a4-9907-3ca6869821c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:36.115700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-5610710e-2380-44a4-9907-3ca6869821c1 None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:36.116963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5610710e-2380-44a4-9907-3ca6869821c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:36.124480 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1211/2381] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 3 12:56:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:36.136868 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c62589-ad27-4e40-961a-fb8eaffe1aaf None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:36.151119 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c62589-ad27-4e40-961a-fb8eaffe1aaf None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:36.156313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa83530-0866-4c38-90b8-45ca710a1c76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:36.156933 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa83530-0866-4c38-90b8-45ca710a1c76 service swift] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:36.157211 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa83530-0866-4c38-90b8-45ca710a1c76 service swift] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:36.157476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa83530-0866-4c38-90b8-45ca710a1c76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:36.167419 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa83530-0866-4c38-90b8-45ca710a1c76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:36.168994 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa83530-0866-4c38-90b8-45ca710a1c76 service swift] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:36.179267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1171/2382] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:56:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 12:56:46.109855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:46.175662 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=67 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:51.780430 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01946581-ec8c-43ff-8b26-7edbc9a602b6 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:51.780966 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01946581-ec8c-43ff-8b26-7edbc9a602b6 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:51.781076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01946581-ec8c-43ff-8b26-7edbc9a602b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:51.801546 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-01946581-ec8c-43ff-8b26-7edbc9a602b6 None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:51.802798 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-01946581-ec8c-43ff-8b26-7edbc9a602b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:51.810114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1212/2383] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 3 12:56:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:51.822674 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa83530-0866-4c38-90b8-45ca710a1c76 service swift] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:51.833928 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa83530-0866-4c38-90b8-45ca710a1c76 service swift] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:51.839294 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b3362-6994-4e50-a284-3b487c092198 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:51.839965 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b3362-6994-4e50-a284-3b487c092198 service swift] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:51.840250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b3362-6994-4e50-a284-3b487c092198 service swift] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:51.840525 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b3362-6994-4e50-a284-3b487c092198 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:51.849066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b3362-6994-4e50-a284-3b487c092198 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:51.850708 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b3362-6994-4e50-a284-3b487c092198 service swift] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:51.862967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1172/2384] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:56:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 03 12:56:52.419144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dcdf2-dfd7-4d9f-80dc-5d79f4ce92d5 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:52.419502 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dcdf2-dfd7-4d9f-80dc-5d79f4ce92d5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:52.419837 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826dcdf2-dfd7-4d9f-80dc-5d79f4ce92d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:52.440034 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-826dcdf2-dfd7-4d9f-80dc-5d79f4ce92d5 None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:52.441291 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-826dcdf2-dfd7-4d9f-80dc-5d79f4ce92d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:52.448650 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1213/2385] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 3 12:56:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:52.460655 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b3362-6994-4e50-a284-3b487c092198 service swift] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:52.471054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b3362-6994-4e50-a284-3b487c092198 service swift] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:52.475983 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794219da-421e-45dd-9029-f2ad66a6f0c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:52.476578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794219da-421e-45dd-9029-f2ad66a6f0c6 service swift] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:52.477032 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794219da-421e-45dd-9029-f2ad66a6f0c6 service swift] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:52.477303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794219da-421e-45dd-9029-f2ad66a6f0c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:52.485679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794219da-421e-45dd-9029-f2ad66a6f0c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:52.487827 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794219da-421e-45dd-9029-f2ad66a6f0c6 service swift] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:52.496218 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1173/2386] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:56:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 12:56:52.709386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c43428-2abc-457d-a497-e0ce566becc3 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:52.709674 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c43428-2abc-457d-a497-e0ce566becc3 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:52.709950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c43428-2abc-457d-a497-e0ce566becc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:52.728964 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-08c43428-2abc-457d-a497-e0ce566becc3 None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:52.730146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-08c43428-2abc-457d-a497-e0ce566becc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:52.737823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1214/2387] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 3 12:56:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:52.749698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794219da-421e-45dd-9029-f2ad66a6f0c6 service swift] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:52.760313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794219da-421e-45dd-9029-f2ad66a6f0c6 service swift] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:52.765355 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436a6f8-9b3a-428e-b41d-f10579816883 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:52.765954 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436a6f8-9b3a-428e-b41d-f10579816883 service swift] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:52.766238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436a6f8-9b3a-428e-b41d-f10579816883 service swift] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:52.766566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436a6f8-9b3a-428e-b41d-f10579816883 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:52.774449 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436a6f8-9b3a-428e-b41d-f10579816883 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:52.776050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436a6f8-9b3a-428e-b41d-f10579816883 service swift] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:52.784754 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1174/2388] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:56:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 03 12:56:53.116979 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65f379-6ded-43f9-bff6-23b28c52de8b None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:53.117659 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65f379-6ded-43f9-bff6-23b28c52de8b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:53.117659 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65f379-6ded-43f9-bff6-23b28c52de8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:53.160931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-2f65f379-6ded-43f9-bff6-23b28c52de8b None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:53.162184 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-2f65f379-6ded-43f9-bff6-23b28c52de8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:53.194253 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1215/2389] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 3 12:56:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:53.211265 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436a6f8-9b3a-428e-b41d-f10579816883 service swift] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:53.222887 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436a6f8-9b3a-428e-b41d-f10579816883 service swift] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:53.228602 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be71186-8d40-429b-a027-797b626118b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:53.229230 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be71186-8d40-429b-a027-797b626118b9 service swift] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:53.229512 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be71186-8d40-429b-a027-797b626118b9 service swift] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:53.229791 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be71186-8d40-429b-a027-797b626118b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:53.244731 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be71186-8d40-429b-a027-797b626118b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:53.246373 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be71186-8d40-429b-a027-797b626118b9 service swift] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:53.255554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1175/2390] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:56:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 12:56:54.056755 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e05309-9a79-4b40-b810-246e54bbc245 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:54.057044 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e05309-9a79-4b40-b810-246e54bbc245 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:54.057372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e05309-9a79-4b40-b810-246e54bbc245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:54.234258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-62e05309-9a79-4b40-b810-246e54bbc245 None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:54.235500 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-62e05309-9a79-4b40-b810-246e54bbc245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:54.243959 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1216/2391] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 3 12:56:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:54.259630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be71186-8d40-429b-a027-797b626118b9 service swift] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:54.272039 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be71186-8d40-429b-a027-797b626118b9 service swift] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:54.279578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7dea1e-e28f-4967-8471-f3c5a3da5e98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:54.280413 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7dea1e-e28f-4967-8471-f3c5a3da5e98 service swift] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:54.280848 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7dea1e-e28f-4967-8471-f3c5a3da5e98 service swift] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:54.281293 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7dea1e-e28f-4967-8471-f3c5a3da5e98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:54.304747 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7dea1e-e28f-4967-8471-f3c5a3da5e98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:54.306360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7dea1e-e28f-4967-8471-f3c5a3da5e98 service swift] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:54.315449 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1176/2392] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:56:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 12:56:55.442675 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e05309-9a79-4b40-b810-246e54bbc245 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:55.513597 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e05309-9a79-4b40-b810-246e54bbc245 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:55.518618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe74f92-078d-46da-b509-825523df55ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:55.519252 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe74f92-078d-46da-b509-825523df55ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:55.519530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe74f92-078d-46da-b509-825523df55ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:55.519801 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe74f92-078d-46da-b509-825523df55ff admin admin] PATH_INFO: `/v3/users/381f1faddc1b4905b92ebba918213df4` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:55.520948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe74f92-078d-46da-b509-825523df55ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=381f1faddc1b4905b92ebba918213df4)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:55.522873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe74f92-078d-46da-b509-825523df55ff admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:55.568318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bfe74f92-078d-46da-b509-825523df55ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '381f1faddc1b4905b92ebba918213df4'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:55.576971 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bfe74f92-078d-46da-b509-825523df55ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '381f1faddc1b4905b92ebba918213df4'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:55.590308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bfe74f92-078d-46da-b509-825523df55ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '381f1faddc1b4905b92ebba918213df4'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:55.591000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bfe74f92-078d-46da-b509-825523df55ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '381f1faddc1b4905b92ebba918213df4'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:55.591623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bfe74f92-078d-46da-b509-825523df55ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '381f1faddc1b4905b92ebba918213df4'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:55.604749 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1217/2393] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:56:55 2022] DELETE /identity/v3/users/381f1faddc1b4905b92ebba918213df4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:56:55.609394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5116c89-9965-4629-8633-54e8818228c3 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:55.609807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5116c89-9965-4629-8633-54e8818228c3 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:55.610009 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5116c89-9965-4629-8633-54e8818228c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:55.630088 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-f5116c89-9965-4629-8633-54e8818228c3 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:56:55.631334 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f5116c89-9965-4629-8633-54e8818228c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:55.692476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1177/2394] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:56:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:56:55.701867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe74f92-078d-46da-b509-825523df55ff admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:55.703370 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-bfe74f92-078d-46da-b509-825523df55ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:55.756358 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:56:55.761359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe74f92-078d-46da-b509-825523df55ff admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:55.771734 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010b899b-35bd-486f-aebe-c4097be35243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:55.772356 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b899b-35bd-486f-aebe-c4097be35243 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:55.772635 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b899b-35bd-486f-aebe-c4097be35243 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:55.772906 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010b899b-35bd-486f-aebe-c4097be35243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:55.782476 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010b899b-35bd-486f-aebe-c4097be35243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:55.784416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010b899b-35bd-486f-aebe-c4097be35243 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:55.793804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1218/2395] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:56:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:56:56.064210 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5116c89-9965-4629-8633-54e8818228c3 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:56:56.065813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f5116c89-9965-4629-8633-54e8818228c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:56:56.078229 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5116c89-9965-4629-8633-54e8818228c3 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:56:56.083675 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a1473-2d4a-45fd-9791-560983e8a4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:56:56.084262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a1473-2d4a-45fd-9791-560983e8a4f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:56:56.084544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a1473-2d4a-45fd-9791-560983e8a4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:56:56.084820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a1473-2d4a-45fd-9791-560983e8a4f6 admin admin] PATH_INFO: `/v3/projects/39cef51c667e481c8f385d547d6a57da` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:56:56.085979 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a1473-2d4a-45fd-9791-560983e8a4f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39cef51c667e481c8f385d547d6a57da)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:56:56.088112 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a1473-2d4a-45fd-9791-560983e8a4f6 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:56:56.133865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-164a1473-2d4a-45fd-9791-560983e8a4f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39cef51c667e481c8f385d547d6a57da', 'request_id': 'req-164a1473-2d4a-45fd-9791-560983e8a4f6'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:56.140885 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-164a1473-2d4a-45fd-9791-560983e8a4f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39cef51c667e481c8f385d547d6a57da', 'request_id': 'req-164a1473-2d4a-45fd-9791-560983e8a4f6'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:56.147510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-164a1473-2d4a-45fd-9791-560983e8a4f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39cef51c667e481c8f385d547d6a57da', 'request_id': 'req-164a1473-2d4a-45fd-9791-560983e8a4f6'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:56.157563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-164a1473-2d4a-45fd-9791-560983e8a4f6 admin admin] The token cache is being invalidate because project 39cef51c667e481c8f385d547d6a57da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:56:56.157851 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-164a1473-2d4a-45fd-9791-560983e8a4f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:56.159136 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-164a1473-2d4a-45fd-9791-560983e8a4f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:56:56.160627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1178/2396] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:56:56 2022] DELETE /identity/v3/projects/39cef51c667e481c8f385d547d6a57da => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:57:05.789638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=5 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:57:06.154680 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:57:07.186159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010b899b-35bd-486f-aebe-c4097be35243 service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:57:07.187859 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-010b899b-35bd-486f-aebe-c4097be35243 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:57:07.200035 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010b899b-35bd-486f-aebe-c4097be35243 service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:57:07.251933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd620ad-c84a-483c-a1fb-ec5d7cc318db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:57:07.252707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd620ad-c84a-483c-a1fb-ec5d7cc318db service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:57:07.252991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd620ad-c84a-483c-a1fb-ec5d7cc318db service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:57:07.253266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd620ad-c84a-483c-a1fb-ec5d7cc318db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:57:07.255213 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8fd620ad-c84a-483c-a1fb-ec5d7cc318db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:57:07.320873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd620ad-c84a-483c-a1fb-ec5d7cc318db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:57:07.322292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd620ad-c84a-483c-a1fb-ec5d7cc318db service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:57:07.332085 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1219/2397] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:57:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:57:14.788756 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d59d16-c2e7-4e54-bead-cff57610c5a0 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:57:14.789289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d59d16-c2e7-4e54-bead-cff57610c5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:57:14.789289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d59d16-c2e7-4e54-bead-cff57610c5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:57:14.808475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-06d59d16-c2e7-4e54-bead-cff57610c5a0 None None] MFA Rules not processed for user `44fbd50cb5664149814abf0e9e44e18f`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:57:14.809664 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-06d59d16-c2e7-4e54-bead-cff57610c5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:57:14.863717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1179/2398] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:57:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:57:17.327585 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=50 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:57:24.855620 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=25 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:57:46.490850 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4b7fb-7a00-4861-92dc-3c218c221276 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:57:46.491378 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4b7fb-7a00-4861-92dc-3c218c221276 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:57:46.491478 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4b7fb-7a00-4861-92dc-3c218c221276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:57:46.510225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-68e4b7fb-7a00-4861-92dc-3c218c221276 None None] MFA Rules not processed for user `13128464bb2547f5838a787d022e6c5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:57:46.511445 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-68e4b7fb-7a00-4861-92dc-3c218c221276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:57:46.564763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1220/2399] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:57:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:57:54.802845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358f947-4125-48a3-858b-8e2cf2d37837 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:57:54.803380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358f947-4125-48a3-858b-8e2cf2d37837 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:57:54.803380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f358f947-4125-48a3-858b-8e2cf2d37837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:57:54.823553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-f358f947-4125-48a3-858b-8e2cf2d37837 None None] MFA Rules not processed for user `5f16ff28aac54514a475f8a4f5c6e4c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:57:54.824766 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f358f947-4125-48a3-858b-8e2cf2d37837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:57:54.880178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1180/2400] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:57:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:57:55.625794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4b7fb-7a00-4861-92dc-3c218c221276 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:57:55.627239 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-68e4b7fb-7a00-4861-92dc-3c218c221276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:57:55.639108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4b7fb-7a00-4861-92dc-3c218c221276 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:57:55.642798 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575385a-4f56-4212-be88-bef6e9b610cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:57:55.643414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575385a-4f56-4212-be88-bef6e9b610cc service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:57:55.643694 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575385a-4f56-4212-be88-bef6e9b610cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:57:55.643967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575385a-4f56-4212-be88-bef6e9b610cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:57:55.646351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-6575385a-4f56-4212-be88-bef6e9b610cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:57:55.738884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575385a-4f56-4212-be88-bef6e9b610cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:57:55.740253 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575385a-4f56-4212-be88-bef6e9b610cc service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:57:55.756845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1221/2401] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:57:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:57:58.812271 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358f947-4125-48a3-858b-8e2cf2d37837 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:57:58.813965 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f358f947-4125-48a3-858b-8e2cf2d37837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:57:58.829370 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f358f947-4125-48a3-858b-8e2cf2d37837 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:57:58.833538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4490dce-746a-4389-8d28-06eed18bd579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:57:58.834702 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4490dce-746a-4389-8d28-06eed18bd579 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:57:58.835182 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4490dce-746a-4389-8d28-06eed18bd579 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:57:58.835667 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4490dce-746a-4389-8d28-06eed18bd579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:57:58.837903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a4490dce-746a-4389-8d28-06eed18bd579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:57:58.853516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4490dce-746a-4389-8d28-06eed18bd579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:57:58.855112 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4490dce-746a-4389-8d28-06eed18bd579 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:57:58.870645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1181/2402] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:57:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:58:05.749289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=59 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:08.867699 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=31 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:12.224281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575385a-4f56-4212-be88-bef6e9b610cc service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:12.225807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-6575385a-4f56-4212-be88-bef6e9b610cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:12.282274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575385a-4f56-4212-be88-bef6e9b610cc service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:12.286417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd16ef-511e-4465-b6da-6941c68794d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:12.287052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd16ef-511e-4465-b6da-6941c68794d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:12.287325 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd16ef-511e-4465-b6da-6941c68794d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:12.287592 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd16ef-511e-4465-b6da-6941c68794d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:12.289319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-26fd16ef-511e-4465-b6da-6941c68794d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:12.300639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd16ef-511e-4465-b6da-6941c68794d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:12.302040 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd16ef-511e-4465-b6da-6941c68794d4 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:12.311080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1222/2403] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:17.233247 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc521e3-8159-4e8e-92d5-483d456ac0c5 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:17.233748 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc521e3-8159-4e8e-92d5-483d456ac0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:17.233844 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc521e3-8159-4e8e-92d5-483d456ac0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:17.255191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-2cc521e3-8159-4e8e-92d5-483d456ac0c5 None None] MFA Rules not processed for user `8ccd455b417549e3b00439cee4577d11`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:17.256405 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2cc521e3-8159-4e8e-92d5-483d456ac0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:17.311540 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1182/2404] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:17.630387 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd16ef-511e-4465-b6da-6941c68794d4 service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:17.639460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd16ef-511e-4465-b6da-6941c68794d4 service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:17.643002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f9424-ecd6-4579-b86b-1024a53b8fa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:17.643617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f9424-ecd6-4579-b86b-1024a53b8fa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:17.643876 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f9424-ecd6-4579-b86b-1024a53b8fa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:17.644147 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f9424-ecd6-4579-b86b-1024a53b8fa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:17.645887 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e27f9424-ecd6-4579-b86b-1024a53b8fa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:17.655530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f9424-ecd6-4579-b86b-1024a53b8fa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:17.657352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f9424-ecd6-4579-b86b-1024a53b8fa7 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:17.666169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1223/2405] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:18.099123 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc521e3-8159-4e8e-92d5-483d456ac0c5 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:18.100631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2cc521e3-8159-4e8e-92d5-483d456ac0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:18.171975 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc521e3-8159-4e8e-92d5-483d456ac0c5 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:18.177122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68365862-719c-4198-94bf-70d536d0fb31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5094263ac471481db48783b19192e01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:18.177728 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68365862-719c-4198-94bf-70d536d0fb31 service placement] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:18.178005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68365862-719c-4198-94bf-70d536d0fb31 service placement] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:18.178270 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68365862-719c-4198-94bf-70d536d0fb31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:18.180011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-68365862-719c-4198-94bf-70d536d0fb31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:18.191067 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68365862-719c-4198-94bf-70d536d0fb31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:18.192624 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68365862-719c-4198-94bf-70d536d0fb31 service placement] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:18.202170 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1183/2406] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:24.284932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f9424-ecd6-4579-b86b-1024a53b8fa7 service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:24.293533 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f9424-ecd6-4579-b86b-1024a53b8fa7 service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:24.298304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6a68d-5c5f-4fdb-b211-d2ba2e84a3af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:24.298952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6a68d-5c5f-4fdb-b211-d2ba2e84a3af service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:24.299225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6a68d-5c5f-4fdb-b211-d2ba2e84a3af service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:24.299491 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6a68d-5c5f-4fdb-b211-d2ba2e84a3af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:24.301244 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-31c6a68d-5c5f-4fdb-b211-d2ba2e84a3af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:24.361857 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6a68d-5c5f-4fdb-b211-d2ba2e84a3af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:24.363248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6a68d-5c5f-4fdb-b211-d2ba2e84a3af service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:24.373172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1224/2407] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:27.750648 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68365862-719c-4198-94bf-70d536d0fb31 service placement] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:27.759110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-68365862-719c-4198-94bf-70d536d0fb31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:27.772015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68365862-719c-4198-94bf-70d536d0fb31 service placement] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:27.777075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cef22c-c974-46ad-9db7-d4fa1d5bfbb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5094263ac471481db48783b19192e01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:27.777683 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cef22c-c974-46ad-9db7-d4fa1d5bfbb2 service placement] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:27.777964 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cef22c-c974-46ad-9db7-d4fa1d5bfbb2 service placement] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:27.778262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cef22c-c974-46ad-9db7-d4fa1d5bfbb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:27.780005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-63cef22c-c974-46ad-9db7-d4fa1d5bfbb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:27.794131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cef22c-c974-46ad-9db7-d4fa1d5bfbb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:27.795792 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cef22c-c974-46ad-9db7-d4fa1d5bfbb2 service placement] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:27.813076 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1184/2408] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:27.814534 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6a68d-5c5f-4fdb-b211-d2ba2e84a3af service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:27.816601 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-31c6a68d-5c5f-4fdb-b211-d2ba2e84a3af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:27.829706 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6a68d-5c5f-4fdb-b211-d2ba2e84a3af service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:27.834847 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b0df86-7da0-43b0-a018-6ecb02ae93c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:27.835582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0df86-7da0-43b0-a018-6ecb02ae93c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:27.835872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0df86-7da0-43b0-a018-6ecb02ae93c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:27.836144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b0df86-7da0-43b0-a018-6ecb02ae93c8 admin admin] PATH_INFO: `/v3/users/44fbd50cb5664149814abf0e9e44e18f` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:27.837304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0df86-7da0-43b0-a018-6ecb02ae93c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44fbd50cb5664149814abf0e9e44e18f)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:27.839230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b0df86-7da0-43b0-a018-6ecb02ae93c8 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:27.886808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d8b0df86-7da0-43b0-a018-6ecb02ae93c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44fbd50cb5664149814abf0e9e44e18f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:27.893710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d8b0df86-7da0-43b0-a018-6ecb02ae93c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44fbd50cb5664149814abf0e9e44e18f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:27.905085 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d8b0df86-7da0-43b0-a018-6ecb02ae93c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44fbd50cb5664149814abf0e9e44e18f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:27.905747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d8b0df86-7da0-43b0-a018-6ecb02ae93c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44fbd50cb5664149814abf0e9e44e18f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:27.906417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d8b0df86-7da0-43b0-a018-6ecb02ae93c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44fbd50cb5664149814abf0e9e44e18f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:27.910405 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:27.920513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1225/2409] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:58:27 2022] DELETE /identity/v3/users/44fbd50cb5664149814abf0e9e44e18f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:28.098689 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cef22c-c974-46ad-9db7-d4fa1d5bfbb2 service placement] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:28.109473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cef22c-c974-46ad-9db7-d4fa1d5bfbb2 service placement] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:28.165303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e86d2b-1176-4fab-997f-6c2606e69c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:28.165940 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e86d2b-1176-4fab-997f-6c2606e69c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:28.166216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e86d2b-1176-4fab-997f-6c2606e69c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:28.166504 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e86d2b-1176-4fab-997f-6c2606e69c5a admin admin] PATH_INFO: `/v3/projects/983ea94ea58e41ccbbab9bebe0e89e3c` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:28.167655 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e86d2b-1176-4fab-997f-6c2606e69c5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=983ea94ea58e41ccbbab9bebe0e89e3c)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:28.169531 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e86d2b-1176-4fab-997f-6c2606e69c5a admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:28.193124 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:28.215167 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f1e86d2b-1176-4fab-997f-6c2606e69c5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983ea94ea58e41ccbbab9bebe0e89e3c', 'request_id': 'req-f1e86d2b-1176-4fab-997f-6c2606e69c5a'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:28.222965 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f1e86d2b-1176-4fab-997f-6c2606e69c5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983ea94ea58e41ccbbab9bebe0e89e3c', 'request_id': 'req-f1e86d2b-1176-4fab-997f-6c2606e69c5a'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:28.228916 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f1e86d2b-1176-4fab-997f-6c2606e69c5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983ea94ea58e41ccbbab9bebe0e89e3c', 'request_id': 'req-f1e86d2b-1176-4fab-997f-6c2606e69c5a'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:28.239874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f1e86d2b-1176-4fab-997f-6c2606e69c5a admin admin] The token cache is being invalidate because project 983ea94ea58e41ccbbab9bebe0e89e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:58:28.240163 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f1e86d2b-1176-4fab-997f-6c2606e69c5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:28.240792 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f1e86d2b-1176-4fab-997f-6c2606e69c5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:28.242306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1185/2410] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:58:28 2022] DELETE /identity/v3/projects/983ea94ea58e41ccbbab9bebe0e89e3c => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:28.247286 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6292ab3-feda-4f07-96a7-f41a2add17c9 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:28.247574 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6292ab3-feda-4f07-96a7-f41a2add17c9 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:28.247851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6292ab3-feda-4f07-96a7-f41a2add17c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:28.266661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-f6292ab3-feda-4f07-96a7-f41a2add17c9 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:28.267843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f6292ab3-feda-4f07-96a7-f41a2add17c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:28.326013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1226/2411] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:28.334101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabdf6c-6f7c-4e41-b1ce-f78c17e2f074 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:28.334383 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabdf6c-6f7c-4e41-b1ce-f78c17e2f074 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:28.334690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabdf6c-6f7c-4e41-b1ce-f78c17e2f074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:28.353692 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-9dabdf6c-6f7c-4e41-b1ce-f78c17e2f074 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:28.354915 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-9dabdf6c-6f7c-4e41-b1ce-f78c17e2f074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:28.362600 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1186/2412] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:28.366632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6292ab3-feda-4f07-96a7-f41a2add17c9 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:28.376319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6292ab3-feda-4f07-96a7-f41a2add17c9 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:28.381371 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589de66-fce9-48d7-b258-9a808a1243bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:28.381985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589de66-fce9-48d7-b258-9a808a1243bf admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:28.382246 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589de66-fce9-48d7-b258-9a808a1243bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:28.382552 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589de66-fce9-48d7-b258-9a808a1243bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:28.383248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589de66-fce9-48d7-b258-9a808a1243bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:58:28.383553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589de66-fce9-48d7-b258-9a808a1243bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:28.385133 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589de66-fce9-48d7-b258-9a808a1243bf admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:28.397047 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1227/2413] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:58:28.401037 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabdf6c-6f7c-4e41-b1ce-f78c17e2f074 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:28.410302 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabdf6c-6f7c-4e41-b1ce-f78c17e2f074 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:28.414880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e11751-a59a-41ad-bc6b-9e9505bfd4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:28.415472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e11751-a59a-41ad-bc6b-9e9505bfd4bf admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:28.415748 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e11751-a59a-41ad-bc6b-9e9505bfd4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:28.416017 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e11751-a59a-41ad-bc6b-9e9505bfd4bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:28.416679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e11751-a59a-41ad-bc6b-9e9505bfd4bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:28.418595 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e11751-a59a-41ad-bc6b-9e9505bfd4bf admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:28.445532 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1187/2414] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:58:28 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:58:28.449624 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589de66-fce9-48d7-b258-9a808a1243bf admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:28.460126 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589de66-fce9-48d7-b258-9a808a1243bf admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:28.516154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9373c1-3bd0-4ee9-9558-9a26beaf9a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:28.516764 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9373c1-3bd0-4ee9-9558-9a26beaf9a86 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:28.517041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9373c1-3bd0-4ee9-9558-9a26beaf9a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:28.517312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9373c1-3bd0-4ee9-9558-9a26beaf9a86 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:28.517964 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9373c1-3bd0-4ee9-9558-9a26beaf9a86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:28.519927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9373c1-3bd0-4ee9-9558-9a26beaf9a86 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:28.540231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1228/2415] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:58:28 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:58:28.544241 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e11751-a59a-41ad-bc6b-9e9505bfd4bf admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:28.555432 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e11751-a59a-41ad-bc6b-9e9505bfd4bf admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:28.560015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7c8c3-8826-4eb5-ab9a-ca30597e68ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:28.560620 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7c8c3-8826-4eb5-ab9a-ca30597e68ad admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:28.560902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7c8c3-8826-4eb5-ab9a-ca30597e68ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:28.561178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d7c8c3-8826-4eb5-ab9a-ca30597e68ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:28.561874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7c8c3-8826-4eb5-ab9a-ca30597e68ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:28.567137 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d7c8c3-8826-4eb5-ab9a-ca30597e68ad admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:28.578450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1188/2416] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:58:28.582266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9373c1-3bd0-4ee9-9558-9a26beaf9a86 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:28.592853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9373c1-3bd0-4ee9-9558-9a26beaf9a86 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:28.597550 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a8f6e-ae68-47d3-938f-8708afa3aa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:28.598164 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a8f6e-ae68-47d3-938f-8708afa3aa5b admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:28.598936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a8f6e-ae68-47d3-938f-8708afa3aa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:28.598936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a8f6e-ae68-47d3-938f-8708afa3aa5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:28.599451 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a8f6e-ae68-47d3-938f-8708afa3aa5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:28.601064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a8f6e-ae68-47d3-938f-8708afa3aa5b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:28.611610 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1229/2417] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:58:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:58:28.616338 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7c8c3-8826-4eb5-ab9a-ca30597e68ad admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:28.629405 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d7c8c3-8826-4eb5-ab9a-ca30597e68ad admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:28.634286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f5ec9-86d2-403e-a811-2facedcad5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:28.634989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f5ec9-86d2-403e-a811-2facedcad5d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:28.635307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f5ec9-86d2-403e-a811-2facedcad5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:28.635590 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f5ec9-86d2-403e-a811-2facedcad5d5 admin admin] PATH_INFO: `/v3/projects/013562df792549308bdd556dfd3324ab/users/8b190947b9ff4a86a166ae1522d80893/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:28.651227 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f5ec9-86d2-403e-a811-2facedcad5d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=013562df792549308bdd556dfd3324ab, user_id=8b190947b9ff4a86a166ae1522d80893, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:28.653894 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f5ec9-86d2-403e-a811-2facedcad5d5 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:28.666993 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1189/2418] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:58:28 2022] PUT /identity/v3/projects/013562df792549308bdd556dfd3324ab/users/8b190947b9ff4a86a166ae1522d80893/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:28.678843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80b7db-c088-4346-b8b0-d822672668d2 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:28.679145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80b7db-c088-4346-b8b0-d822672668d2 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:28.679398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80b7db-c088-4346-b8b0-d822672668d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:28.708354 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-3a80b7db-c088-4346-b8b0-d822672668d2 None None] MFA Rules not processed for user `8b190947b9ff4a86a166ae1522d80893`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:28.709625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-3a80b7db-c088-4346-b8b0-d822672668d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:28.757330 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:28.774225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1230/2419] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:28.786431 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f5ec9-86d2-403e-a811-2facedcad5d5 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:28.788232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-176f5ec9-86d2-403e-a811-2facedcad5d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:28.860950 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f5ec9-86d2-403e-a811-2facedcad5d5 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:28.867115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144203e-afc1-450b-93ff-9337f6ce0f16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:28.867733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144203e-afc1-450b-93ff-9337f6ce0f16 service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:28.868012 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144203e-afc1-450b-93ff-9337f6ce0f16 service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:28.868283 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144203e-afc1-450b-93ff-9337f6ce0f16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:28.876349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144203e-afc1-450b-93ff-9337f6ce0f16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:28.877950 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144203e-afc1-450b-93ff-9337f6ce0f16 service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:28.886712 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1190/2420] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:36.123033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80b7db-c088-4346-b8b0-d822672668d2 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:36.185134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80b7db-c088-4346-b8b0-d822672668d2 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:36.189669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58225f-373f-45e3-a8a8-841dd890c139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:36.190301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58225f-373f-45e3-a8a8-841dd890c139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:36.190611 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58225f-373f-45e3-a8a8-841dd890c139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:36.190882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58225f-373f-45e3-a8a8-841dd890c139 admin admin] PATH_INFO: `/v3/users/8b190947b9ff4a86a166ae1522d80893` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:36.192041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58225f-373f-45e3-a8a8-841dd890c139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b190947b9ff4a86a166ae1522d80893)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:36.193938 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58225f-373f-45e3-a8a8-841dd890c139 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:36.240166 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-af58225f-373f-45e3-a8a8-841dd890c139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b190947b9ff4a86a166ae1522d80893'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:36.246957 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-af58225f-373f-45e3-a8a8-841dd890c139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b190947b9ff4a86a166ae1522d80893'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:36.258388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-af58225f-373f-45e3-a8a8-841dd890c139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b190947b9ff4a86a166ae1522d80893'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:36.259079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-af58225f-373f-45e3-a8a8-841dd890c139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b190947b9ff4a86a166ae1522d80893'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:36.259680 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-af58225f-373f-45e3-a8a8-841dd890c139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b190947b9ff4a86a166ae1522d80893'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:36.270510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144203e-afc1-450b-93ff-9337f6ce0f16 service nova] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:36.272021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8144203e-afc1-450b-93ff-9337f6ce0f16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:36.273434 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1231/2421] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:58:36 2022] DELETE /identity/v3/users/8b190947b9ff4a86a166ae1522d80893 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:36.277725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286dceb-5cbc-4731-b793-336609405947 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:36.278036 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286dceb-5cbc-4731-b793-336609405947 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:36.278313 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286dceb-5cbc-4731-b793-336609405947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:36.287772 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144203e-afc1-450b-93ff-9337f6ce0f16 service nova] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:36.298628 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-8286dceb-5cbc-4731-b793-336609405947 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:36.299883 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8286dceb-5cbc-4731-b793-336609405947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:36.317307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:36.353803 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad16aae-0abb-480c-9ece-7d734d5fc6da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:36.354529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad16aae-0abb-480c-9ece-7d734d5fc6da service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:36.354816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad16aae-0abb-480c-9ece-7d734d5fc6da service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:36.355090 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad16aae-0abb-480c-9ece-7d734d5fc6da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:36.356958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-aad16aae-0abb-480c-9ece-7d734d5fc6da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:36.360095 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1232/2422] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:58:36.369669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286dceb-5cbc-4731-b793-336609405947 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:36.371867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8286dceb-5cbc-4731-b793-336609405947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:36.385283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:36.423448 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad16aae-0abb-480c-9ece-7d734d5fc6da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:36.425062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad16aae-0abb-480c-9ece-7d734d5fc6da service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:36.430465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286dceb-5cbc-4731-b793-336609405947 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:36.434056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41e16e-26f1-4bea-bedf-24ca8501b9e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:36.434330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1191/2423] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:36 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:36.434700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41e16e-26f1-4bea-bedf-24ca8501b9e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:36.434977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41e16e-26f1-4bea-bedf-24ca8501b9e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:36.435250 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41e16e-26f1-4bea-bedf-24ca8501b9e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:36.445217 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41e16e-26f1-4bea-bedf-24ca8501b9e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:36.446655 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41e16e-26f1-4bea-bedf-24ca8501b9e9 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:36.457134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1233/2424] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:58:36.737716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad16aae-0abb-480c-9ece-7d734d5fc6da service nova] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:36.739576 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-aad16aae-0abb-480c-9ece-7d734d5fc6da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:36.756321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad16aae-0abb-480c-9ece-7d734d5fc6da service nova] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:36.766191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0813dbe8-ac57-40a6-8913-e026e8ede8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:36.766897 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813dbe8-ac57-40a6-8913-e026e8ede8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:36.767177 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813dbe8-ac57-40a6-8913-e026e8ede8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:36.767454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0813dbe8-ac57-40a6-8913-e026e8ede8c6 admin admin] PATH_INFO: `/v3/projects/013562df792549308bdd556dfd3324ab` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:36.769188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813dbe8-ac57-40a6-8913-e026e8ede8c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=013562df792549308bdd556dfd3324ab)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:36.775807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0813dbe8-ac57-40a6-8913-e026e8ede8c6 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:36.832319 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0813dbe8-ac57-40a6-8913-e026e8ede8c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '013562df792549308bdd556dfd3324ab', 'request_id': 'req-0813dbe8-ac57-40a6-8913-e026e8ede8c6'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:36.840883 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0813dbe8-ac57-40a6-8913-e026e8ede8c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '013562df792549308bdd556dfd3324ab', 'request_id': 'req-0813dbe8-ac57-40a6-8913-e026e8ede8c6'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:36.848686 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0813dbe8-ac57-40a6-8913-e026e8ede8c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '013562df792549308bdd556dfd3324ab', 'request_id': 'req-0813dbe8-ac57-40a6-8913-e026e8ede8c6'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:36.859989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0813dbe8-ac57-40a6-8913-e026e8ede8c6 admin admin] The token cache is being invalidate because project 013562df792549308bdd556dfd3324ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:58:36.860306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0813dbe8-ac57-40a6-8913-e026e8ede8c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:36.861085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0813dbe8-ac57-40a6-8913-e026e8ede8c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:36.863513 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1192/2425] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:58:36 2022] DELETE /identity/v3/projects/013562df792549308bdd556dfd3324ab => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:36.868921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6d64f-0969-4327-b7e0-775576393180 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:36.869493 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6d64f-0969-4327-b7e0-775576393180 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:36.869775 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6d64f-0969-4327-b7e0-775576393180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:36.893266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-a5c6d64f-0969-4327-b7e0-775576393180 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:36.894511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a5c6d64f-0969-4327-b7e0-775576393180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:36.962379 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1234/2426] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:58:36.970356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef65a81-d3c7-4219-b49d-3fa8f8f88bc1 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:36.970679 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef65a81-d3c7-4219-b49d-3fa8f8f88bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:36.970955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef65a81-d3c7-4219-b49d-3fa8f8f88bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:36.991001 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-eef65a81-d3c7-4219-b49d-3fa8f8f88bc1 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:36.992292 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-eef65a81-d3c7-4219-b49d-3fa8f8f88bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:37.000366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1193/2427] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:37.005215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6d64f-0969-4327-b7e0-775576393180 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:37.014757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6d64f-0969-4327-b7e0-775576393180 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:37.019363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5011762-1069-4676-a8a5-ae23079c7b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:37.019963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5011762-1069-4676-a8a5-ae23079c7b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:37.020242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5011762-1069-4676-a8a5-ae23079c7b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:37.020515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5011762-1069-4676-a8a5-ae23079c7b78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:37.021203 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5011762-1069-4676-a8a5-ae23079c7b78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:58:37.021503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5011762-1069-4676-a8a5-ae23079c7b78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:37.023125 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5011762-1069-4676-a8a5-ae23079c7b78 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:37.035312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1235/2428] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:58:37.039633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef65a81-d3c7-4219-b49d-3fa8f8f88bc1 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:37.049675 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef65a81-d3c7-4219-b49d-3fa8f8f88bc1 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:37.054349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0c02e-625c-4cff-b1f8-4aeb3b8731d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:37.054990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0c02e-625c-4cff-b1f8-4aeb3b8731d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:37.055269 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0c02e-625c-4cff-b1f8-4aeb3b8731d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:37.055539 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0c02e-625c-4cff-b1f8-4aeb3b8731d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:37.056202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0c02e-625c-4cff-b1f8-4aeb3b8731d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:37.058066 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0c02e-625c-4cff-b1f8-4aeb3b8731d0 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:37.085832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1194/2429] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:58:37 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:58:37.089945 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5011762-1069-4676-a8a5-ae23079c7b78 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:37.100373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5011762-1069-4676-a8a5-ae23079c7b78 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:37.161195 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fb88e-1d4e-477d-b562-5a233a21efa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:37.162535 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fb88e-1d4e-477d-b562-5a233a21efa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:37.162821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fb88e-1d4e-477d-b562-5a233a21efa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:37.163091 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fb88e-1d4e-477d-b562-5a233a21efa0 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:37.163821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975fb88e-1d4e-477d-b562-5a233a21efa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:37.165753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975fb88e-1d4e-477d-b562-5a233a21efa0 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:37.187478 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1236/2430] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:58:37 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:58:37.191667 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0c02e-625c-4cff-b1f8-4aeb3b8731d0 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:37.201854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0c02e-625c-4cff-b1f8-4aeb3b8731d0 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:37.206507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffb086-6016-4611-9b84-288f39a588bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:37.207065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffb086-6016-4611-9b84-288f39a588bc admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:37.207360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffb086-6016-4611-9b84-288f39a588bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:37.207632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffb086-6016-4611-9b84-288f39a588bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:37.208321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ffb086-6016-4611-9b84-288f39a588bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:37.209903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ffb086-6016-4611-9b84-288f39a588bc admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:37.219894 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1195/2431] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:58:37.224718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fb88e-1d4e-477d-b562-5a233a21efa0 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:37.234701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fb88e-1d4e-477d-b562-5a233a21efa0 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:37.239896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b4f24-173f-417d-9465-bd163277b91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:37.240651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b4f24-173f-417d-9465-bd163277b91a admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:37.240851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b4f24-173f-417d-9465-bd163277b91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:37.241162 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b4f24-173f-417d-9465-bd163277b91a admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:37.241918 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b4f24-173f-417d-9465-bd163277b91a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:37.243567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b4f24-173f-417d-9465-bd163277b91a admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:37.253783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1237/2432] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:58:37.258032 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffb086-6016-4611-9b84-288f39a588bc admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:37.268137 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ffb086-6016-4611-9b84-288f39a588bc admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:37.272778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf5a5d-774f-41e4-8328-0cdd2a256c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:37.273433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf5a5d-774f-41e4-8328-0cdd2a256c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:37.273716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf5a5d-774f-41e4-8328-0cdd2a256c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:37.273991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf5a5d-774f-41e4-8328-0cdd2a256c73 admin admin] PATH_INFO: `/v3/projects/09ec95c3685a46b7a378dccf58c916c4/users/05a0ee0808c740e28b7d813223512501/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:37.285430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:37.289903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf5a5d-774f-41e4-8328-0cdd2a256c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09ec95c3685a46b7a378dccf58c916c4, user_id=05a0ee0808c740e28b7d813223512501, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:37.293026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf5a5d-774f-41e4-8328-0cdd2a256c73 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:37.306798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1196/2433] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:58:37 2022] PUT /identity/v3/projects/09ec95c3685a46b7a378dccf58c916c4/users/05a0ee0808c740e28b7d813223512501/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:37.317084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e58a68-59c7-4049-8270-32da0c6cbc30 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:37.317372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e58a68-59c7-4049-8270-32da0c6cbc30 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:37.317644 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e58a68-59c7-4049-8270-32da0c6cbc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:37.344373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-30e58a68-59c7-4049-8270-32da0c6cbc30 None None] MFA Rules not processed for user `05a0ee0808c740e28b7d813223512501`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:37.345565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-30e58a68-59c7-4049-8270-32da0c6cbc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:37.353976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:37.411265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1238/2434] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:37.415926 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf5a5d-774f-41e4-8328-0cdd2a256c73 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:37.478228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf5a5d-774f-41e4-8328-0cdd2a256c73 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:37.482931 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcf5b4-3431-494f-9e81-1203b9e16b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:37.483543 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcf5b4-3431-494f-9e81-1203b9e16b05 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:37.483824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcf5b4-3431-494f-9e81-1203b9e16b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:37.484107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dcf5b4-3431-494f-9e81-1203b9e16b05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:37.484776 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcf5b4-3431-494f-9e81-1203b9e16b05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:37.487544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dcf5b4-3431-494f-9e81-1203b9e16b05 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:37.514071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1197/2435] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:58:37 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:58:37.518936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e58a68-59c7-4049-8270-32da0c6cbc30 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:37.529618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e58a68-59c7-4049-8270-32da0c6cbc30 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:37.584773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9a02c-b4a7-4351-9a9a-c9d526cca4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:37.585383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9a02c-b4a7-4351-9a9a-c9d526cca4c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:37.585664 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9a02c-b4a7-4351-9a9a-c9d526cca4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:37.585936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9a02c-b4a7-4351-9a9a-c9d526cca4c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:37.586620 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9a02c-b4a7-4351-9a9a-c9d526cca4c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:37.588507 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9a02c-b4a7-4351-9a9a-c9d526cca4c1 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:37.608729 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1239/2436] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:58:37 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:58:37.612799 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcf5b4-3431-494f-9e81-1203b9e16b05 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:37.623111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dcf5b4-3431-494f-9e81-1203b9e16b05 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:37.627981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b04be-c4c4-45b8-ac46-bc3597d80769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:37.628585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b04be-c4c4-45b8-ac46-bc3597d80769 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:37.628866 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b04be-c4c4-45b8-ac46-bc3597d80769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:37.629140 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b04be-c4c4-45b8-ac46-bc3597d80769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:37.629837 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b04be-c4c4-45b8-ac46-bc3597d80769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:37.631454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b04be-c4c4-45b8-ac46-bc3597d80769 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:37.642132 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1198/2437] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:58:37.646704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9a02c-b4a7-4351-9a9a-c9d526cca4c1 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:37.656696 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9a02c-b4a7-4351-9a9a-c9d526cca4c1 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:37.662116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5983d6-c156-41f8-b25d-8bfb131e9f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:37.662813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5983d6-c156-41f8-b25d-8bfb131e9f46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:37.663099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5983d6-c156-41f8-b25d-8bfb131e9f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:37.663373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5983d6-c156-41f8-b25d-8bfb131e9f46 admin admin] PATH_INFO: `/v3/projects/14f3c1f9428d4dc7aac4c33e5385b4b2/users/53210e351dca433ab58b87e2fd06e2a5/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:37.678125 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5983d6-c156-41f8-b25d-8bfb131e9f46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f3c1f9428d4dc7aac4c33e5385b4b2, user_id=53210e351dca433ab58b87e2fd06e2a5, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:37.680785 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5983d6-c156-41f8-b25d-8bfb131e9f46 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:37.693006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1240/2438] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:58:37 2022] PUT /identity/v3/projects/14f3c1f9428d4dc7aac4c33e5385b4b2/users/53210e351dca433ab58b87e2fd06e2a5/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:37.696884 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b04be-c4c4-45b8-ac46-bc3597d80769 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:37.707452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b04be-c4c4-45b8-ac46-bc3597d80769 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:37.768813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e283d-48f9-42b7-af62-dc734e7af6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:37.769449 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e283d-48f9-42b7-af62-dc734e7af6de admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:37.769726 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e283d-48f9-42b7-af62-dc734e7af6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:37.769996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e283d-48f9-42b7-af62-dc734e7af6de admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:37.770854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e283d-48f9-42b7-af62-dc734e7af6de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:37.772666 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e283d-48f9-42b7-af62-dc734e7af6de admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:37.785146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1199/2439] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:58:37 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:58:37.789560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5983d6-c156-41f8-b25d-8bfb131e9f46 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:37.800390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5983d6-c156-41f8-b25d-8bfb131e9f46 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:37.806930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1031cfc-3210-46ae-a39c-832089afb673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:37.807721 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1031cfc-3210-46ae-a39c-832089afb673 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:37.808006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1031cfc-3210-46ae-a39c-832089afb673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:37.808283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1031cfc-3210-46ae-a39c-832089afb673 admin admin] PATH_INFO: `/v3/projects/14f3c1f9428d4dc7aac4c33e5385b4b2/users/53210e351dca433ab58b87e2fd06e2a5/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:37.810704 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1031cfc-3210-46ae-a39c-832089afb673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f3c1f9428d4dc7aac4c33e5385b4b2, user_id=53210e351dca433ab58b87e2fd06e2a5, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:37.813714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1031cfc-3210-46ae-a39c-832089afb673 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:37.825090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1241/2440] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:58:37 2022] PUT /identity/v3/projects/14f3c1f9428d4dc7aac4c33e5385b4b2/users/53210e351dca433ab58b87e2fd06e2a5/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:37.837632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd5867-da31-44ba-8e6c-d769462f4d23 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:37.837928 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd5867-da31-44ba-8e6c-d769462f4d23 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:37.838208 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd5867-da31-44ba-8e6c-d769462f4d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:37.867410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-e2bd5867-da31-44ba-8e6c-d769462f4d23 None None] MFA Rules not processed for user `53210e351dca433ab58b87e2fd06e2a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:37.868623 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e2bd5867-da31-44ba-8e6c-d769462f4d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:37.915375 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:37.937365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1200/2441] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:37 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:37.949627 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1031cfc-3210-46ae-a39c-832089afb673 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:37.951356 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f1031cfc-3210-46ae-a39c-832089afb673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:38.008435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1031cfc-3210-46ae-a39c-832089afb673 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:38.012978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d272576-44e8-4399-8561-04a6e8ec7e38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:38.013595 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d272576-44e8-4399-8561-04a6e8ec7e38 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:38.013868 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d272576-44e8-4399-8561-04a6e8ec7e38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:38.014135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d272576-44e8-4399-8561-04a6e8ec7e38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:38.024332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d272576-44e8-4399-8561-04a6e8ec7e38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:38.025751 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d272576-44e8-4399-8561-04a6e8ec7e38 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:38.035877 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1242/2442] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:37 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:38.140140 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd5867-da31-44ba-8e6c-d769462f4d23 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:38.150925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd5867-da31-44ba-8e6c-d769462f4d23 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:38.154470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33478de5-4dc3-4d1f-974c-5ae07e4ac368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:38.155167 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33478de5-4dc3-4d1f-974c-5ae07e4ac368 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:38.155656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33478de5-4dc3-4d1f-974c-5ae07e4ac368 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:38.155924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33478de5-4dc3-4d1f-974c-5ae07e4ac368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:38.212468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33478de5-4dc3-4d1f-974c-5ae07e4ac368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:38.213999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33478de5-4dc3-4d1f-974c-5ae07e4ac368 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:38.223373 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1201/2443] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:40.110256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d272576-44e8-4399-8561-04a6e8ec7e38 service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:40.112401 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-4d272576-44e8-4399-8561-04a6e8ec7e38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:40.120187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:40.184133 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d272576-44e8-4399-8561-04a6e8ec7e38 service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:40.187855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8128c82-67dd-492f-b04b-48c1b2385bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:40.188459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8128c82-67dd-492f-b04b-48c1b2385bce service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:40.188733 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8128c82-67dd-492f-b04b-48c1b2385bce service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:40.188997 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8128c82-67dd-492f-b04b-48c1b2385bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:40.191066 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d8128c82-67dd-492f-b04b-48c1b2385bce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:40.200741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8128c82-67dd-492f-b04b-48c1b2385bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:40.202141 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8128c82-67dd-492f-b04b-48c1b2385bce service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:40.210464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1243/2444] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 12:58:40.843253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33478de5-4dc3-4d1f-974c-5ae07e4ac368 service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:40.916140 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33478de5-4dc3-4d1f-974c-5ae07e4ac368 service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:40.919943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1177f-086e-4529-bacc-150499e5f856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:40.920656 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1177f-086e-4529-bacc-150499e5f856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:40.921009 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1177f-086e-4529-bacc-150499e5f856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:40.921359 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1177f-086e-4529-bacc-150499e5f856 admin admin] PATH_INFO: `/v3/users/05a0ee0808c740e28b7d813223512501` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:40.923490 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1177f-086e-4529-bacc-150499e5f856 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05a0ee0808c740e28b7d813223512501)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:40.925698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e1177f-086e-4529-bacc-150499e5f856 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:40.978644 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-a4e1177f-086e-4529-bacc-150499e5f856 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05a0ee0808c740e28b7d813223512501'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:40.997562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-a4e1177f-086e-4529-bacc-150499e5f856 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05a0ee0808c740e28b7d813223512501'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:41.006741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-a4e1177f-086e-4529-bacc-150499e5f856 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05a0ee0808c740e28b7d813223512501'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:41.007361 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-a4e1177f-086e-4529-bacc-150499e5f856 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05a0ee0808c740e28b7d813223512501'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:41.019997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-a4e1177f-086e-4529-bacc-150499e5f856 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05a0ee0808c740e28b7d813223512501'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:41.028690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1202/2445] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:58:40 2022] DELETE /identity/v3/users/05a0ee0808c740e28b7d813223512501 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:41.033109 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c72d59-77ee-4a9a-b6d9-7485dc6888de None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:41.033402 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c72d59-77ee-4a9a-b6d9-7485dc6888de None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:41.033722 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c72d59-77ee-4a9a-b6d9-7485dc6888de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:41.060814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-c4c72d59-77ee-4a9a-b6d9-7485dc6888de None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:41.062158 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c4c72d59-77ee-4a9a-b6d9-7485dc6888de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:41.120752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1244/2446] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:41.130943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1177f-086e-4529-bacc-150499e5f856 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:41.132369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a4e1177f-086e-4529-bacc-150499e5f856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:41.137738 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:41.193562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e1177f-086e-4529-bacc-150499e5f856 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:41.197293 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe53b7-1937-4aa7-aa26-7671dbe95dae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:41.197912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe53b7-1937-4aa7-aa26-7671dbe95dae service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:41.198191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe53b7-1937-4aa7-aa26-7671dbe95dae service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:41.198463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe53b7-1937-4aa7-aa26-7671dbe95dae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:41.208698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe53b7-1937-4aa7-aa26-7671dbe95dae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:41.210106 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affe53b7-1937-4aa7-aa26-7671dbe95dae service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:41.219583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1203/2447] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:41.482994 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c72d59-77ee-4a9a-b6d9-7485dc6888de None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:41.484583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c4c72d59-77ee-4a9a-b6d9-7485dc6888de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:41.499547 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c72d59-77ee-4a9a-b6d9-7485dc6888de None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:41.504840 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:41.505474 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:41.505830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:41.506126 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161 admin admin] PATH_INFO: `/v3/projects/09ec95c3685a46b7a378dccf58c916c4` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:41.507325 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09ec95c3685a46b7a378dccf58c916c4)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:41.509204 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:41.556808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09ec95c3685a46b7a378dccf58c916c4', 'request_id': 'req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:41.563750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09ec95c3685a46b7a378dccf58c916c4', 'request_id': 'req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:41.570555 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09ec95c3685a46b7a378dccf58c916c4', 'request_id': 'req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:41.580999 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161 admin admin] The token cache is being invalidate because project 09ec95c3685a46b7a378dccf58c916c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:58:41.581286 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:41.581898 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:41.583408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1245/2448] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:58:41 2022] DELETE /identity/v3/projects/09ec95c3685a46b7a378dccf58c916c4 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:41.587457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe53b7-1937-4aa7-aa26-7671dbe95dae service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:41.597485 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe53b7-1937-4aa7-aa26-7671dbe95dae service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:41.652167 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387a02f-e9e2-4662-b1bc-ea8c99164420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:41.652802 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387a02f-e9e2-4662-b1bc-ea8c99164420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:41.653078 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387a02f-e9e2-4662-b1bc-ea8c99164420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:41.653352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387a02f-e9e2-4662-b1bc-ea8c99164420 admin admin] PATH_INFO: `/v3/users/53210e351dca433ab58b87e2fd06e2a5` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:41.654525 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387a02f-e9e2-4662-b1bc-ea8c99164420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53210e351dca433ab58b87e2fd06e2a5)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:41.656454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387a02f-e9e2-4662-b1bc-ea8c99164420 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:41.702933 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-4387a02f-e9e2-4662-b1bc-ea8c99164420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53210e351dca433ab58b87e2fd06e2a5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:41.714378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-4387a02f-e9e2-4662-b1bc-ea8c99164420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53210e351dca433ab58b87e2fd06e2a5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:41.720728 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-4387a02f-e9e2-4662-b1bc-ea8c99164420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53210e351dca433ab58b87e2fd06e2a5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:41.721827 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-4387a02f-e9e2-4662-b1bc-ea8c99164420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53210e351dca433ab58b87e2fd06e2a5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:41.728976 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-4387a02f-e9e2-4662-b1bc-ea8c99164420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53210e351dca433ab58b87e2fd06e2a5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:41.734875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1204/2449] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:58:41 2022] DELETE /identity/v3/users/53210e351dca433ab58b87e2fd06e2a5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:41.999305 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:42.000843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:42.014559 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a4627-cf45-4857-a3fe-f0f2ab6a4161 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:42.045097 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:42.074198 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1e7f3-ae58-4111-ba04-6aa578eaa345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:42.074868 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1e7f3-ae58-4111-ba04-6aa578eaa345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:42.075146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1e7f3-ae58-4111-ba04-6aa578eaa345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:42.075418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1e7f3-ae58-4111-ba04-6aa578eaa345 admin admin] PATH_INFO: `/v3/projects/14f3c1f9428d4dc7aac4c33e5385b4b2` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:42.076608 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1e7f3-ae58-4111-ba04-6aa578eaa345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14f3c1f9428d4dc7aac4c33e5385b4b2)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:42.078455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1e7f3-ae58-4111-ba04-6aa578eaa345 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:42.128404 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-11c1e7f3-ae58-4111-ba04-6aa578eaa345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14f3c1f9428d4dc7aac4c33e5385b4b2', 'request_id': 'req-11c1e7f3-ae58-4111-ba04-6aa578eaa345'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:42.135703 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-11c1e7f3-ae58-4111-ba04-6aa578eaa345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14f3c1f9428d4dc7aac4c33e5385b4b2', 'request_id': 'req-11c1e7f3-ae58-4111-ba04-6aa578eaa345'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:42.143951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-11c1e7f3-ae58-4111-ba04-6aa578eaa345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14f3c1f9428d4dc7aac4c33e5385b4b2', 'request_id': 'req-11c1e7f3-ae58-4111-ba04-6aa578eaa345'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:42.156246 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-11c1e7f3-ae58-4111-ba04-6aa578eaa345 admin admin] The token cache is being invalidate because project 14f3c1f9428d4dc7aac4c33e5385b4b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:58:42.165171 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-11c1e7f3-ae58-4111-ba04-6aa578eaa345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:42.166797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-11c1e7f3-ae58-4111-ba04-6aa578eaa345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:42.169110 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1246/2450] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:58:41 2022] DELETE /identity/v3/projects/14f3c1f9428d4dc7aac4c33e5385b4b2 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:42.174286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9229f1-929b-40dd-942b-af37f94640a7 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:42.174887 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9229f1-929b-40dd-942b-af37f94640a7 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:42.174887 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9229f1-929b-40dd-942b-af37f94640a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:42.194158 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-ce9229f1-929b-40dd-942b-af37f94640a7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:42.195402 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ce9229f1-929b-40dd-942b-af37f94640a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:42.215476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:42.259187 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1205/2451] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:42.266923 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221cf09-d8c7-488a-a8f4-7dca095abda7 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:42.267307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221cf09-d8c7-488a-a8f4-7dca095abda7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:42.267647 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221cf09-d8c7-488a-a8f4-7dca095abda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:42.287948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-d221cf09-d8c7-488a-a8f4-7dca095abda7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:42.289124 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d221cf09-d8c7-488a-a8f4-7dca095abda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:42.296987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1247/2452] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:42.301283 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9229f1-929b-40dd-942b-af37f94640a7 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:42.310972 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9229f1-929b-40dd-942b-af37f94640a7 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:42.315547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d515af-d510-45d4-b32a-edf70f1de89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:42.316156 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d515af-d510-45d4-b32a-edf70f1de89e admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:42.316440 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d515af-d510-45d4-b32a-edf70f1de89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:42.316717 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d515af-d510-45d4-b32a-edf70f1de89e admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:42.317416 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d515af-d510-45d4-b32a-edf70f1de89e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:58:42.317722 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d515af-d510-45d4-b32a-edf70f1de89e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:42.319351 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d515af-d510-45d4-b32a-edf70f1de89e admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:42.331733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1206/2453] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:58:42.335219 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221cf09-d8c7-488a-a8f4-7dca095abda7 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:42.344144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221cf09-d8c7-488a-a8f4-7dca095abda7 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:42.349517 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39d217-8d95-478c-9807-cd2c7e4cd217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:42.350121 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39d217-8d95-478c-9807-cd2c7e4cd217 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:42.350401 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39d217-8d95-478c-9807-cd2c7e4cd217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:42.350716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39d217-8d95-478c-9807-cd2c7e4cd217 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:42.351373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39d217-8d95-478c-9807-cd2c7e4cd217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:42.353279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39d217-8d95-478c-9807-cd2c7e4cd217 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:42.381774 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1248/2454] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:58:42 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:58:42.383477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d515af-d510-45d4-b32a-edf70f1de89e admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:42.394213 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d515af-d510-45d4-b32a-edf70f1de89e admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:42.449430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad50450-ae66-429a-af82-c8dc3aaf389d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:42.450040 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad50450-ae66-429a-af82-c8dc3aaf389d admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:42.450341 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad50450-ae66-429a-af82-c8dc3aaf389d admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:42.450628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad50450-ae66-429a-af82-c8dc3aaf389d admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:42.451295 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad50450-ae66-429a-af82-c8dc3aaf389d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:42.453210 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad50450-ae66-429a-af82-c8dc3aaf389d admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:42.474278 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1207/2455] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:58:42 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:58:42.478404 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39d217-8d95-478c-9807-cd2c7e4cd217 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:42.488569 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39d217-8d95-478c-9807-cd2c7e4cd217 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:42.493393 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7669c6-f67b-497d-ba6b-de0d5d41ac92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:42.494152 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7669c6-f67b-497d-ba6b-de0d5d41ac92 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:42.494432 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7669c6-f67b-497d-ba6b-de0d5d41ac92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:42.494740 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7669c6-f67b-497d-ba6b-de0d5d41ac92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:42.495437 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7669c6-f67b-497d-ba6b-de0d5d41ac92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:42.497029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7669c6-f67b-497d-ba6b-de0d5d41ac92 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:42.507642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1249/2456] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:58:42.511412 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad50450-ae66-429a-af82-c8dc3aaf389d admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:42.522452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad50450-ae66-429a-af82-c8dc3aaf389d admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:42.527358 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f674f-1fcd-4580-b7d8-eb80251aadc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:42.527995 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f674f-1fcd-4580-b7d8-eb80251aadc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:42.528275 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f674f-1fcd-4580-b7d8-eb80251aadc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:42.528550 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f674f-1fcd-4580-b7d8-eb80251aadc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:42.529256 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f674f-1fcd-4580-b7d8-eb80251aadc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:42.530930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f674f-1fcd-4580-b7d8-eb80251aadc7 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:42.541489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1208/2457] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:58:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:58:42.545508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7669c6-f67b-497d-ba6b-de0d5d41ac92 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:42.555605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7669c6-f67b-497d-ba6b-de0d5d41ac92 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:42.560225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90d2ac-0abe-4421-bf87-244e3be66f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:42.560868 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90d2ac-0abe-4421-bf87-244e3be66f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:42.561153 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90d2ac-0abe-4421-bf87-244e3be66f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:42.561428 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90d2ac-0abe-4421-bf87-244e3be66f45 admin admin] PATH_INFO: `/v3/projects/6b3878bad04b4d81b59a8309e7a1f7fc/users/5d0f1d942b684c4094ad7b4c29d91721/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:42.576136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90d2ac-0abe-4421-bf87-244e3be66f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b3878bad04b4d81b59a8309e7a1f7fc, user_id=5d0f1d942b684c4094ad7b4c29d91721, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:42.578784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90d2ac-0abe-4421-bf87-244e3be66f45 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:42.592374 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1250/2458] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:58:42 2022] PUT /identity/v3/projects/6b3878bad04b4d81b59a8309e7a1f7fc/users/5d0f1d942b684c4094ad7b4c29d91721/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:42.602388 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1aeb9e-a10d-48a5-8f05-aaec47f9d1c6 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:42.602702 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1aeb9e-a10d-48a5-8f05-aaec47f9d1c6 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:42.602980 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1aeb9e-a10d-48a5-8f05-aaec47f9d1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:42.630145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-ff1aeb9e-a10d-48a5-8f05-aaec47f9d1c6 None None] MFA Rules not processed for user `5d0f1d942b684c4094ad7b4c29d91721`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:42.631511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ff1aeb9e-a10d-48a5-8f05-aaec47f9d1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:42.695700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1209/2459] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:42.707185 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90d2ac-0abe-4421-bf87-244e3be66f45 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:42.708843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8d90d2ac-0abe-4421-bf87-244e3be66f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:42.764278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90d2ac-0abe-4421-bf87-244e3be66f45 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:42.767816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d5c70-5d88-4b35-b9d8-fef2ca70504f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:42.768412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d5c70-5d88-4b35-b9d8-fef2ca70504f service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:42.768688 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d5c70-5d88-4b35-b9d8-fef2ca70504f service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:42.768957 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d5c70-5d88-4b35-b9d8-fef2ca70504f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:42.776967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d5c70-5d88-4b35-b9d8-fef2ca70504f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:42.778391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d5c70-5d88-4b35-b9d8-fef2ca70504f service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:42.782813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:42.787928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1251/2460] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:42.809492 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1aeb9e-a10d-48a5-8f05-aaec47f9d1c6 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:42.830758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:42.873180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1aeb9e-a10d-48a5-8f05-aaec47f9d1c6 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:42.879049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4a466-d9c3-458c-91e9-01f4d587a0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:42.879798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4a466-d9c3-458c-91e9-01f4d587a0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:42.879962 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4a466-d9c3-458c-91e9-01f4d587a0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:42.880237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4a466-d9c3-458c-91e9-01f4d587a0e2 admin admin] PATH_INFO: `/v3/users/5d0f1d942b684c4094ad7b4c29d91721` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:42.881466 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4a466-d9c3-458c-91e9-01f4d587a0e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d0f1d942b684c4094ad7b4c29d91721)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:42.883504 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4a466-d9c3-458c-91e9-01f4d587a0e2 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:42.929645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ffa4a466-d9c3-458c-91e9-01f4d587a0e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d0f1d942b684c4094ad7b4c29d91721'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:42.942922 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ffa4a466-d9c3-458c-91e9-01f4d587a0e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d0f1d942b684c4094ad7b4c29d91721'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:42.950493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ffa4a466-d9c3-458c-91e9-01f4d587a0e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d0f1d942b684c4094ad7b4c29d91721'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:42.951141 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ffa4a466-d9c3-458c-91e9-01f4d587a0e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d0f1d942b684c4094ad7b4c29d91721'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:42.958689 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ffa4a466-d9c3-458c-91e9-01f4d587a0e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d0f1d942b684c4094ad7b4c29d91721'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:42.964622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1210/2461] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:58:42 2022] DELETE /identity/v3/users/5d0f1d942b684c4094ad7b4c29d91721 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:42.969115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b099f8-73b9-4471-8973-9bea25ec5591 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:42.969417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b099f8-73b9-4471-8973-9bea25ec5591 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:42.969694 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b099f8-73b9-4471-8973-9bea25ec5591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:42.988960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-b8b099f8-73b9-4471-8973-9bea25ec5591 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:42.990140 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b8b099f8-73b9-4471-8973-9bea25ec5591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:43.056178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1252/2462] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:58:43.066326 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4a466-d9c3-458c-91e9-01f4d587a0e2 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:43.067882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ffa4a466-d9c3-458c-91e9-01f4d587a0e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:43.125552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4a466-d9c3-458c-91e9-01f4d587a0e2 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:43.129907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e089cc2-13b8-437a-bfc7-c4ab293768e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:43.130547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e089cc2-13b8-437a-bfc7-c4ab293768e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:43.130826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e089cc2-13b8-437a-bfc7-c4ab293768e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:43.131096 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e089cc2-13b8-437a-bfc7-c4ab293768e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:43.140179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e089cc2-13b8-437a-bfc7-c4ab293768e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:43.141637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e089cc2-13b8-437a-bfc7-c4ab293768e7 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:43.152101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1211/2463] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:43.503285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b099f8-73b9-4471-8973-9bea25ec5591 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:43.503285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b8b099f8-73b9-4471-8973-9bea25ec5591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:43.515945 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b099f8-73b9-4471-8973-9bea25ec5591 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:43.526808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9781d50-96b7-4f23-8ffb-997a361fbc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:43.527733 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9781d50-96b7-4f23-8ffb-997a361fbc8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:43.528123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9781d50-96b7-4f23-8ffb-997a361fbc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:43.528488 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9781d50-96b7-4f23-8ffb-997a361fbc8f admin admin] PATH_INFO: `/v3/projects/6b3878bad04b4d81b59a8309e7a1f7fc` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:43.529857 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9781d50-96b7-4f23-8ffb-997a361fbc8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b3878bad04b4d81b59a8309e7a1f7fc)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:43.532098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9781d50-96b7-4f23-8ffb-997a361fbc8f admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:43.583713 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a9781d50-96b7-4f23-8ffb-997a361fbc8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b3878bad04b4d81b59a8309e7a1f7fc', 'request_id': 'req-a9781d50-96b7-4f23-8ffb-997a361fbc8f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:43.590873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a9781d50-96b7-4f23-8ffb-997a361fbc8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b3878bad04b4d81b59a8309e7a1f7fc', 'request_id': 'req-a9781d50-96b7-4f23-8ffb-997a361fbc8f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:43.597144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a9781d50-96b7-4f23-8ffb-997a361fbc8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b3878bad04b4d81b59a8309e7a1f7fc', 'request_id': 'req-a9781d50-96b7-4f23-8ffb-997a361fbc8f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:43.607625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a9781d50-96b7-4f23-8ffb-997a361fbc8f admin admin] The token cache is being invalidate because project 6b3878bad04b4d81b59a8309e7a1f7fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:58:43.607910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a9781d50-96b7-4f23-8ffb-997a361fbc8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:43.608535 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a9781d50-96b7-4f23-8ffb-997a361fbc8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:43.610077 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1253/2464] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:58:43 2022] DELETE /identity/v3/projects/6b3878bad04b4d81b59a8309e7a1f7fc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:43.615495 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad7565-dc80-4e89-8a1c-e3d508c0fe13 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:43.615960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad7565-dc80-4e89-8a1c-e3d508c0fe13 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:43.616056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad7565-dc80-4e89-8a1c-e3d508c0fe13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:43.636350 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-7cad7565-dc80-4e89-8a1c-e3d508c0fe13 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:43.637578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-7cad7565-dc80-4e89-8a1c-e3d508c0fe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:43.676307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:43.699490 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1212/2465] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:43.708376 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe9b5b-fe68-45c3-9957-5e2dc761d613 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:43.708660 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe9b5b-fe68-45c3-9957-5e2dc761d613 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:43.708932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe9b5b-fe68-45c3-9957-5e2dc761d613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:43.728127 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-09fe9b5b-fe68-45c3-9957-5e2dc761d613 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:43.729324 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-09fe9b5b-fe68-45c3-9957-5e2dc761d613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:43.737675 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1254/2466] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:43.741607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad7565-dc80-4e89-8a1c-e3d508c0fe13 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:43.751893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad7565-dc80-4e89-8a1c-e3d508c0fe13 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:43.757086 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d86617-fbcb-44b1-84c0-5ae828e31d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:43.757690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d86617-fbcb-44b1-84c0-5ae828e31d2a admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:43.757969 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d86617-fbcb-44b1-84c0-5ae828e31d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:43.758268 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d86617-fbcb-44b1-84c0-5ae828e31d2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:43.758992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d86617-fbcb-44b1-84c0-5ae828e31d2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:58:43.759299 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d86617-fbcb-44b1-84c0-5ae828e31d2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:43.760895 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d86617-fbcb-44b1-84c0-5ae828e31d2a admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:43.772166 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1213/2467] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:58:43.776307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe9b5b-fe68-45c3-9957-5e2dc761d613 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:43.785515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe9b5b-fe68-45c3-9957-5e2dc761d613 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:43.790125 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016e1e0-ffd8-4a38-a67b-4f6ffe157026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:43.790757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016e1e0-ffd8-4a38-a67b-4f6ffe157026 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:43.791060 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016e1e0-ffd8-4a38-a67b-4f6ffe157026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:43.791314 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016e1e0-ffd8-4a38-a67b-4f6ffe157026 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:43.791971 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016e1e0-ffd8-4a38-a67b-4f6ffe157026 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:43.793844 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016e1e0-ffd8-4a38-a67b-4f6ffe157026 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:43.821223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1255/2468] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:58:43 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:58:43.825299 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d86617-fbcb-44b1-84c0-5ae828e31d2a admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:43.835544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d86617-fbcb-44b1-84c0-5ae828e31d2a admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:43.892459 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b94ea-90ef-4737-8326-dd6191c9b6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:43.893106 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b94ea-90ef-4737-8326-dd6191c9b6b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:43.893391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b94ea-90ef-4737-8326-dd6191c9b6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:43.893667 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9b94ea-90ef-4737-8326-dd6191c9b6b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:43.894331 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b94ea-90ef-4737-8326-dd6191c9b6b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:43.896534 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9b94ea-90ef-4737-8326-dd6191c9b6b9 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:43.917588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1214/2469] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:58:43 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:58:43.922503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016e1e0-ffd8-4a38-a67b-4f6ffe157026 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:43.933566 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016e1e0-ffd8-4a38-a67b-4f6ffe157026 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:43.938105 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb5138-dfa0-45fd-a118-e81e07e5621e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:43.938738 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb5138-dfa0-45fd-a118-e81e07e5621e admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:43.939020 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb5138-dfa0-45fd-a118-e81e07e5621e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:43.939294 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb5138-dfa0-45fd-a118-e81e07e5621e admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:43.940009 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb5138-dfa0-45fd-a118-e81e07e5621e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:43.941600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fb5138-dfa0-45fd-a118-e81e07e5621e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:43.951268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1256/2470] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:58:43.954912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b94ea-90ef-4737-8326-dd6191c9b6b9 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:43.964859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9b94ea-90ef-4737-8326-dd6191c9b6b9 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:43.969578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbaf1e-a048-49b6-9e4b-b5a01c2a4474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:43.970186 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbaf1e-a048-49b6-9e4b-b5a01c2a4474 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:43.970468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbaf1e-a048-49b6-9e4b-b5a01c2a4474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:43.970773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbaf1e-a048-49b6-9e4b-b5a01c2a4474 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:43.971990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbaf1e-a048-49b6-9e4b-b5a01c2a4474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:43.973583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbaf1e-a048-49b6-9e4b-b5a01c2a4474 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:43.984267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1215/2471] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:58:43.988922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb5138-dfa0-45fd-a118-e81e07e5621e admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:43.998580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb5138-dfa0-45fd-a118-e81e07e5621e admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:44.003155 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc75006-1ea7-4c2b-b7bf-0d8d70950999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:44.003796 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc75006-1ea7-4c2b-b7bf-0d8d70950999 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:44.004071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc75006-1ea7-4c2b-b7bf-0d8d70950999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:44.004340 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc75006-1ea7-4c2b-b7bf-0d8d70950999 admin admin] PATH_INFO: `/v3/projects/d479fc0cb11142488f4f336fe1c24840/users/1fd41b379a79441d8d8320ac6c7ac54a/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:44.018618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc75006-1ea7-4c2b-b7bf-0d8d70950999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d479fc0cb11142488f4f336fe1c24840, user_id=1fd41b379a79441d8d8320ac6c7ac54a, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:44.021197 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc75006-1ea7-4c2b-b7bf-0d8d70950999 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:44.034955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1257/2472] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:58:43 2022] PUT /identity/v3/projects/d479fc0cb11142488f4f336fe1c24840/users/1fd41b379a79441d8d8320ac6c7ac54a/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:44.045298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb9fe3-0669-40e0-b6b7-3e2c7ca7dc59 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:44.045583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb9fe3-0669-40e0-b6b7-3e2c7ca7dc59 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:44.045863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb9fe3-0669-40e0-b6b7-3e2c7ca7dc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:44.074445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-f8cb9fe3-0669-40e0-b6b7-3e2c7ca7dc59 None None] MFA Rules not processed for user `1fd41b379a79441d8d8320ac6c7ac54a`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:44.075675 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f8cb9fe3-0669-40e0-b6b7-3e2c7ca7dc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:44.138321 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1216/2473] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:44.148814 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc75006-1ea7-4c2b-b7bf-0d8d70950999 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:44.150458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-7cc75006-1ea7-4c2b-b7bf-0d8d70950999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:44.171825 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:44.208464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc75006-1ea7-4c2b-b7bf-0d8d70950999 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:44.212133 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713a6c9-a71e-43df-a503-1624b276b4bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:44.212740 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713a6c9-a71e-43df-a503-1624b276b4bd service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:44.213029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713a6c9-a71e-43df-a503-1624b276b4bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:44.213302 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5713a6c9-a71e-43df-a503-1624b276b4bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:44.222007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713a6c9-a71e-43df-a503-1624b276b4bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:44.223463 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5713a6c9-a71e-43df-a503-1624b276b4bd service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:44.232131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1258/2474] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:45.719330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb9fe3-0669-40e0-b6b7-3e2c7ca7dc59 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:45.721401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f8cb9fe3-0669-40e0-b6b7-3e2c7ca7dc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:45.782303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:45.826450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb9fe3-0669-40e0-b6b7-3e2c7ca7dc59 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:45.834805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e75d9-4bb6-4a1f-b878-6e253f58fc92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:45.835863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e75d9-4bb6-4a1f-b878-6e253f58fc92 service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:45.836437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e75d9-4bb6-4a1f-b878-6e253f58fc92 service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:45.836728 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e75d9-4bb6-4a1f-b878-6e253f58fc92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:45.839400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d83e75d9-4bb6-4a1f-b878-6e253f58fc92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:45.855355 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e75d9-4bb6-4a1f-b878-6e253f58fc92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:45.857080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e75d9-4bb6-4a1f-b878-6e253f58fc92 service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:45.891561 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1217/2475] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:46.915204 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713a6c9-a71e-43df-a503-1624b276b4bd service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:46.934570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5713a6c9-a71e-43df-a503-1624b276b4bd service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:46.941392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fc97c-85fd-4e8c-922b-a7c9d60838ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:46.942015 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fc97c-85fd-4e8c-922b-a7c9d60838ff service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:46.942298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fc97c-85fd-4e8c-922b-a7c9d60838ff service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:46.942602 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fc97c-85fd-4e8c-922b-a7c9d60838ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:46.944364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f02fc97c-85fd-4e8c-922b-a7c9d60838ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:46.961851 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fc97c-85fd-4e8c-922b-a7c9d60838ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:46.963721 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fc97c-85fd-4e8c-922b-a7c9d60838ff service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:46.976467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1259/2476] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:47.428960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e75d9-4bb6-4a1f-b878-6e253f58fc92 service nova] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:47.491219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e75d9-4bb6-4a1f-b878-6e253f58fc92 service nova] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:47.495950 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3c399-1175-42b3-8561-f6d63e8135b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:47.496592 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3c399-1175-42b3-8561-f6d63e8135b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:47.496874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3c399-1175-42b3-8561-f6d63e8135b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:47.497151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3c399-1175-42b3-8561-f6d63e8135b2 admin admin] PATH_INFO: `/v3/users/1fd41b379a79441d8d8320ac6c7ac54a` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:47.498323 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3c399-1175-42b3-8561-f6d63e8135b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fd41b379a79441d8d8320ac6c7ac54a)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:47.500267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a3c399-1175-42b3-8561-f6d63e8135b2 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:47.553239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-88a3c399-1175-42b3-8561-f6d63e8135b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fd41b379a79441d8d8320ac6c7ac54a'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:47.570265 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-88a3c399-1175-42b3-8561-f6d63e8135b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fd41b379a79441d8d8320ac6c7ac54a'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:47.581052 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-88a3c399-1175-42b3-8561-f6d63e8135b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fd41b379a79441d8d8320ac6c7ac54a'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:47.582072 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-88a3c399-1175-42b3-8561-f6d63e8135b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fd41b379a79441d8d8320ac6c7ac54a'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:47.593488 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-88a3c399-1175-42b3-8561-f6d63e8135b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fd41b379a79441d8d8320ac6c7ac54a'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:47.600847 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1218/2477] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:58:47 2022] DELETE /identity/v3/users/1fd41b379a79441d8d8320ac6c7ac54a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:47.605108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c24ee-779a-47c0-a44b-b51559b326a7 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:47.605398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c24ee-779a-47c0-a44b-b51559b326a7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:47.605676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c24ee-779a-47c0-a44b-b51559b326a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:47.642558 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-a88c24ee-779a-47c0-a44b-b51559b326a7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:47.644428 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a88c24ee-779a-47c0-a44b-b51559b326a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:47.715275 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1260/2478] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:47.725217 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3c399-1175-42b3-8561-f6d63e8135b2 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:47.727227 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-88a3c399-1175-42b3-8561-f6d63e8135b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:47.782796 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:47.798381 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3c399-1175-42b3-8561-f6d63e8135b2 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:47.803101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459bcd2-1e3b-4a8a-a777-552e648a0d56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:47.804176 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459bcd2-1e3b-4a8a-a777-552e648a0d56 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:47.804528 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459bcd2-1e3b-4a8a-a777-552e648a0d56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:47.804759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459bcd2-1e3b-4a8a-a777-552e648a0d56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:47.817239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459bcd2-1e3b-4a8a-a777-552e648a0d56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:47.818843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459bcd2-1e3b-4a8a-a777-552e648a0d56 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:47.831719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1219/2479] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:48.138884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c24ee-779a-47c0-a44b-b51559b326a7 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:48.140337 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a88c24ee-779a-47c0-a44b-b51559b326a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:48.152892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c24ee-779a-47c0-a44b-b51559b326a7 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:48.158801 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71294ffd-272c-4e6c-9c05-ac0345d04238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:48.159806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71294ffd-272c-4e6c-9c05-ac0345d04238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:48.159806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71294ffd-272c-4e6c-9c05-ac0345d04238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:48.159981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71294ffd-272c-4e6c-9c05-ac0345d04238 admin admin] PATH_INFO: `/v3/projects/d479fc0cb11142488f4f336fe1c24840` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:48.161224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71294ffd-272c-4e6c-9c05-ac0345d04238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d479fc0cb11142488f4f336fe1c24840)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:48.163380 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71294ffd-272c-4e6c-9c05-ac0345d04238 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:48.214215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-71294ffd-272c-4e6c-9c05-ac0345d04238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd479fc0cb11142488f4f336fe1c24840', 'request_id': 'req-71294ffd-272c-4e6c-9c05-ac0345d04238'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:48.224683 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-71294ffd-272c-4e6c-9c05-ac0345d04238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd479fc0cb11142488f4f336fe1c24840', 'request_id': 'req-71294ffd-272c-4e6c-9c05-ac0345d04238'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:48.231309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-71294ffd-272c-4e6c-9c05-ac0345d04238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd479fc0cb11142488f4f336fe1c24840', 'request_id': 'req-71294ffd-272c-4e6c-9c05-ac0345d04238'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:48.243795 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-71294ffd-272c-4e6c-9c05-ac0345d04238 admin admin] The token cache is being invalidate because project d479fc0cb11142488f4f336fe1c24840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:58:48.244259 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-71294ffd-272c-4e6c-9c05-ac0345d04238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:48.245033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-71294ffd-272c-4e6c-9c05-ac0345d04238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:58:48.246643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1261/2480] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:58:48 2022] DELETE /identity/v3/projects/d479fc0cb11142488f4f336fe1c24840 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:48.251704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294fe67-0415-4f64-bfb2-6d2f23446e19 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:48.252634 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294fe67-0415-4f64-bfb2-6d2f23446e19 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:48.252986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3294fe67-0415-4f64-bfb2-6d2f23446e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:48.272643 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-3294fe67-0415-4f64-bfb2-6d2f23446e19 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:48.273858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-3294fe67-0415-4f64-bfb2-6d2f23446e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:48.346857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1220/2481] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:48.354855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977fd78-9643-4285-b240-cb46d0fb6536 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:48.355211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977fd78-9643-4285-b240-cb46d0fb6536 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:48.355856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977fd78-9643-4285-b240-cb46d0fb6536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:48.382929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-b977fd78-9643-4285-b240-cb46d0fb6536 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:48.384399 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b977fd78-9643-4285-b240-cb46d0fb6536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:48.399741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1262/2482] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:48.404311 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294fe67-0415-4f64-bfb2-6d2f23446e19 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:48.416467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3294fe67-0415-4f64-bfb2-6d2f23446e19 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:48.423660 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572510e6-9f8b-4a9c-beea-8c20d6251867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:48.424804 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572510e6-9f8b-4a9c-beea-8c20d6251867 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:48.425080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572510e6-9f8b-4a9c-beea-8c20d6251867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:48.425351 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572510e6-9f8b-4a9c-beea-8c20d6251867 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:48.426249 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572510e6-9f8b-4a9c-beea-8c20d6251867 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:58:48.426369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572510e6-9f8b-4a9c-beea-8c20d6251867 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:48.427993 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572510e6-9f8b-4a9c-beea-8c20d6251867 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:48.449394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1221/2483] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:58:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:58:48.450716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977fd78-9643-4285-b240-cb46d0fb6536 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:48.462044 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977fd78-9643-4285-b240-cb46d0fb6536 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:48.471456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21f892-2168-4d71-8635-94c95c631a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:48.472092 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21f892-2168-4d71-8635-94c95c631a87 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:48.472373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21f892-2168-4d71-8635-94c95c631a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:48.472643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21f892-2168-4d71-8635-94c95c631a87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:48.473322 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21f892-2168-4d71-8635-94c95c631a87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:48.475234 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21f892-2168-4d71-8635-94c95c631a87 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:48.503099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1263/2484] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:58:48 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 12:58:48.507281 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572510e6-9f8b-4a9c-beea-8c20d6251867 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:48.517414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572510e6-9f8b-4a9c-beea-8c20d6251867 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:48.573383 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5358ab5-4773-4b74-b824-062cb55a0aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:48.573995 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5358ab5-4773-4b74-b824-062cb55a0aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:48.574274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5358ab5-4773-4b74-b824-062cb55a0aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:48.574574 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5358ab5-4773-4b74-b824-062cb55a0aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:48.575251 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5358ab5-4773-4b74-b824-062cb55a0aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:48.577150 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5358ab5-4773-4b74-b824-062cb55a0aa5 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:48.599166 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1222/2485] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:58:48 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:58:48.603205 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21f892-2168-4d71-8635-94c95c631a87 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:48.609924 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:48.614861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21f892-2168-4d71-8635-94c95c631a87 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:48.620717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946bcba-1cf5-4b22-b376-7fc867549da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:48.621432 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946bcba-1cf5-4b22-b376-7fc867549da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:48.621869 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946bcba-1cf5-4b22-b376-7fc867549da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:48.622373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946bcba-1cf5-4b22-b376-7fc867549da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:48.623231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946bcba-1cf5-4b22-b376-7fc867549da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:48.625056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946bcba-1cf5-4b22-b376-7fc867549da8 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:48.635307 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1264/2486] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:58:48.639949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5358ab5-4773-4b74-b824-062cb55a0aa5 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:48.649950 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5358ab5-4773-4b74-b824-062cb55a0aa5 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:48.654857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9345d7-cb16-414b-a5a1-030bf908a171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:48.655522 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9345d7-cb16-414b-a5a1-030bf908a171 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:48.655783 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9345d7-cb16-414b-a5a1-030bf908a171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:48.656026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9345d7-cb16-414b-a5a1-030bf908a171 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:48.656784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9345d7-cb16-414b-a5a1-030bf908a171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:48.658960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9345d7-cb16-414b-a5a1-030bf908a171 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:48.670164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1223/2487] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:58:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:58:48.674773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946bcba-1cf5-4b22-b376-7fc867549da8 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:48.685442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946bcba-1cf5-4b22-b376-7fc867549da8 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:48.690304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b50ad4-2748-45e5-ab5b-43b2f2f65a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:48.691018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b50ad4-2748-45e5-ab5b-43b2f2f65a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:48.691279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b50ad4-2748-45e5-ab5b-43b2f2f65a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:48.691590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b50ad4-2748-45e5-ab5b-43b2f2f65a73 admin admin] PATH_INFO: `/v3/projects/5f4f8e3907cd44e687d18f5a890ef278/users/6d333c7a05af409f811c4382f048d1fb/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:48.706940 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b50ad4-2748-45e5-ab5b-43b2f2f65a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f4f8e3907cd44e687d18f5a890ef278, user_id=6d333c7a05af409f811c4382f048d1fb, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:48.709795 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b50ad4-2748-45e5-ab5b-43b2f2f65a73 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:48.722594 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1265/2488] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:58:48 2022] PUT /identity/v3/projects/5f4f8e3907cd44e687d18f5a890ef278/users/6d333c7a05af409f811c4382f048d1fb/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:58:48.727663 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a1fbd-26ca-46cc-b1e4-112e934a3220 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:48.728009 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a1fbd-26ca-46cc-b1e4-112e934a3220 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:48.728315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a1fbd-26ca-46cc-b1e4-112e934a3220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:48.749269 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-9a3a1fbd-26ca-46cc-b1e4-112e934a3220 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:48.750519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-9a3a1fbd-26ca-46cc-b1e4-112e934a3220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:48.793988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:48.809711 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1224/2489] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:48.819853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b50ad4-2748-45e5-ab5b-43b2f2f65a73 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:48.821539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-69b50ad4-2748-45e5-ab5b-43b2f2f65a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:48.877771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b50ad4-2748-45e5-ab5b-43b2f2f65a73 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:48.881427 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0459c7-4c6b-41ea-a103-c4f6d9fea6ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:48.882038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0459c7-4c6b-41ea-a103-c4f6d9fea6ba service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:48.882316 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0459c7-4c6b-41ea-a103-c4f6d9fea6ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:48.882625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0459c7-4c6b-41ea-a103-c4f6d9fea6ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:48.891613 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0459c7-4c6b-41ea-a103-c4f6d9fea6ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:48.893061 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0459c7-4c6b-41ea-a103-c4f6d9fea6ba service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:48.902576 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1266/2490] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:49.491120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1776dd3-9169-4294-a1bf-dda87533e595 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:49.491628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1776dd3-9169-4294-a1bf-dda87533e595 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:49.491761 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1776dd3-9169-4294-a1bf-dda87533e595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:49.544460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-d1776dd3-9169-4294-a1bf-dda87533e595 None None] MFA Rules not processed for user `6d333c7a05af409f811c4382f048d1fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:49.546366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d1776dd3-9169-4294-a1bf-dda87533e595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:49.641413 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1225/2491] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:49.651938 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0459c7-4c6b-41ea-a103-c4f6d9fea6ba service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:49.654162 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-bd0459c7-4c6b-41ea-a103-c4f6d9fea6ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:49.725546 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0459c7-4c6b-41ea-a103-c4f6d9fea6ba service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:49.728014 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbd6b4-78fe-4486-ba3e-ef12818f730e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:49.728712 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbd6b4-78fe-4486-ba3e-ef12818f730e service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:49.729057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbd6b4-78fe-4486-ba3e-ef12818f730e service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:49.729433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbd6b4-78fe-4486-ba3e-ef12818f730e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:49.741837 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cbd6b4-78fe-4486-ba3e-ef12818f730e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:49.743322 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cbd6b4-78fe-4486-ba3e-ef12818f730e service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:49.753529 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1267/2492] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:51.110206 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b709b11-bcd6-4cd7-9e4f-e2d94c07ae89 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:51.110748 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b709b11-bcd6-4cd7-9e4f-e2d94c07ae89 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:51.110853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b709b11-bcd6-4cd7-9e4f-e2d94c07ae89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:51.129813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-1b709b11-bcd6-4cd7-9e4f-e2d94c07ae89 None None] MFA Rules not processed for user `6d333c7a05af409f811c4382f048d1fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:51.131025 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1b709b11-bcd6-4cd7-9e4f-e2d94c07ae89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:51.137705 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1226/2493] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:58:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:58:51.368477 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbd6b4-78fe-4486-ba3e-ef12818f730e service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:51.378316 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbd6b4-78fe-4486-ba3e-ef12818f730e service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:51.381874 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a4aeb8-5af9-463a-a330-0a2232189dfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:51.382474 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4aeb8-5af9-463a-a330-0a2232189dfa service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:51.382784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4aeb8-5af9-463a-a330-0a2232189dfa service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:51.383058 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a4aeb8-5af9-463a-a330-0a2232189dfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:51.384796 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-62a4aeb8-5af9-463a-a330-0a2232189dfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:51.395146 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a4aeb8-5af9-463a-a330-0a2232189dfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:51.396613 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a4aeb8-5af9-463a-a330-0a2232189dfa service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:51.405046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1268/2494] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:53.399617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b709b11-bcd6-4cd7-9e4f-e2d94c07ae89 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:53.401048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1b709b11-bcd6-4cd7-9e4f-e2d94c07ae89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:53.493595 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b709b11-bcd6-4cd7-9e4f-e2d94c07ae89 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:53.502923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d967708-9652-45c2-aab1-a950dfa1c01c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:53.503646 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d967708-9652-45c2-aab1-a950dfa1c01c service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:53.503922 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d967708-9652-45c2-aab1-a950dfa1c01c service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:53.504190 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d967708-9652-45c2-aab1-a950dfa1c01c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:53.506072 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-0d967708-9652-45c2-aab1-a950dfa1c01c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:53.529292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9dae6-abd5-475f-b74c-93b9a3ae46e4 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:53.529781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9dae6-abd5-475f-b74c-93b9a3ae46e4 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:53.529949 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9dae6-abd5-475f-b74c-93b9a3ae46e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:53.550782 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-f6c9dae6-abd5-475f-b74c-93b9a3ae46e4 None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:58:53.551974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f6c9dae6-abd5-475f-b74c-93b9a3ae46e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:53.575227 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d967708-9652-45c2-aab1-a950dfa1c01c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:53.576971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d967708-9652-45c2-aab1-a950dfa1c01c service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:53.586888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1227/2495] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:58:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:58:53.637177 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1269/2496] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 3 12:58:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:58:53.688041 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d967708-9652-45c2-aab1-a950dfa1c01c service nova] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:58:53.689985 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-0d967708-9652-45c2-aab1-a950dfa1c01c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:58:53.704387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:58:53.773429 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d967708-9652-45c2-aab1-a950dfa1c01c service nova] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:58:53.778996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9b05e-f3d2-45e6-943c-4a8bb6f99deb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:58:53.779723 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9b05e-f3d2-45e6-943c-4a8bb6f99deb service swift] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:58:53.779994 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9b05e-f3d2-45e6-943c-4a8bb6f99deb service swift] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:58:53.780253 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9b05e-f3d2-45e6-943c-4a8bb6f99deb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:58:53.789298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9b05e-f3d2-45e6-943c-4a8bb6f99deb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:58:53.791064 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9b05e-f3d2-45e6-943c-4a8bb6f99deb service swift] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:58:53.800611 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1228/2497] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:58:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 12:59:01.654590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9dae6-abd5-475f-b74c-93b9a3ae46e4 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:01.683398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9dae6-abd5-475f-b74c-93b9a3ae46e4 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:01.695661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2f332-fc6e-4cf9-b9ec-3e37a4132f91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:01.696482 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2f332-fc6e-4cf9-b9ec-3e37a4132f91 service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:01.696938 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2f332-fc6e-4cf9-b9ec-3e37a4132f91 service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:01.697374 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2f332-fc6e-4cf9-b9ec-3e37a4132f91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:01.700728 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-4ed2f332-fc6e-4cf9-b9ec-3e37a4132f91 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:01.711691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9b05e-f3d2-45e6-943c-4a8bb6f99deb service swift] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:01.713793 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e0f9b05e-f3d2-45e6-943c-4a8bb6f99deb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:01.717454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:59:01.735120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2f332-fc6e-4cf9-b9ec-3e37a4132f91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:01.737554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2f332-fc6e-4cf9-b9ec-3e37a4132f91 service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:01.743864 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9b05e-f3d2-45e6-943c-4a8bb6f99deb service swift] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:01.753970 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1270/2498] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:59:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:59:01.756262 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c68e61-3e3d-431a-b764-79134c1c7370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:01.757122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c68e61-3e3d-431a-b764-79134c1c7370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:01.757578 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c68e61-3e3d-431a-b764-79134c1c7370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:01.758011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c68e61-3e3d-431a-b764-79134c1c7370 admin admin] PATH_INFO: `/v3/users/27136c9cafe24290bfdd4f07fea0320d` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:01.759436 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c68e61-3e3d-431a-b764-79134c1c7370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27136c9cafe24290bfdd4f07fea0320d)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:01.761768 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c68e61-3e3d-431a-b764-79134c1c7370 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:01.811176 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-b7c68e61-3e3d-431a-b764-79134c1c7370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27136c9cafe24290bfdd4f07fea0320d'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:01.824285 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-b7c68e61-3e3d-431a-b764-79134c1c7370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27136c9cafe24290bfdd4f07fea0320d'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:01.832450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-b7c68e61-3e3d-431a-b764-79134c1c7370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27136c9cafe24290bfdd4f07fea0320d'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:01.833095 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-b7c68e61-3e3d-431a-b764-79134c1c7370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27136c9cafe24290bfdd4f07fea0320d'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:01.840128 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-b7c68e61-3e3d-431a-b764-79134c1c7370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27136c9cafe24290bfdd4f07fea0320d'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:01.846883 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1229/2499] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:59:01 2022] DELETE /identity/v3/users/27136c9cafe24290bfdd4f07fea0320d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 12:59:02.087342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2f332-fc6e-4cf9-b9ec-3e37a4132f91 service nova] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:02.107173 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2f332-fc6e-4cf9-b9ec-3e37a4132f91 service nova] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:02.175419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:02.176060 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:02.176337 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:02.176607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619 admin admin] PATH_INFO: `/v3/projects/32d9e8ab388545eebbe8a2b7d60d5c9b` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:02.177775 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32d9e8ab388545eebbe8a2b7d60d5c9b)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:02.180144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:02.225570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32d9e8ab388545eebbe8a2b7d60d5c9b', 'request_id': 'req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:02.232505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32d9e8ab388545eebbe8a2b7d60d5c9b', 'request_id': 'req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:02.239343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32d9e8ab388545eebbe8a2b7d60d5c9b', 'request_id': 'req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:02.249702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619 admin admin] The token cache is being invalidate because project 32d9e8ab388545eebbe8a2b7d60d5c9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:59:02.250505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:02.251162 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:02.252692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1271/2500] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:59:02 2022] DELETE /identity/v3/projects/32d9e8ab388545eebbe8a2b7d60d5c9b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:59:02.256889 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c68e61-3e3d-431a-b764-79134c1c7370 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:02.258301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-b7c68e61-3e3d-431a-b764-79134c1c7370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:02.271746 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c68e61-3e3d-431a-b764-79134c1c7370 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:02.327827 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b17b0-8623-4cf6-a07e-3723f8efbc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:02.328465 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b17b0-8623-4cf6-a07e-3723f8efbc13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:02.328728 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b17b0-8623-4cf6-a07e-3723f8efbc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:02.328999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b17b0-8623-4cf6-a07e-3723f8efbc13 admin admin] PATH_INFO: `/v3/users/99a660a9790f438fa0ee4d376f7e6707` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:02.330688 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b17b0-8623-4cf6-a07e-3723f8efbc13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99a660a9790f438fa0ee4d376f7e6707)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:02.332567 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b17b0-8623-4cf6-a07e-3723f8efbc13 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:02.345277 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:59:02.378414 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-4e8b17b0-8623-4cf6-a07e-3723f8efbc13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99a660a9790f438fa0ee4d376f7e6707'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:02.389852 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-4e8b17b0-8623-4cf6-a07e-3723f8efbc13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99a660a9790f438fa0ee4d376f7e6707'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:02.396701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-4e8b17b0-8623-4cf6-a07e-3723f8efbc13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99a660a9790f438fa0ee4d376f7e6707'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:02.397366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-4e8b17b0-8623-4cf6-a07e-3723f8efbc13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99a660a9790f438fa0ee4d376f7e6707'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:02.404131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-4e8b17b0-8623-4cf6-a07e-3723f8efbc13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99a660a9790f438fa0ee4d376f7e6707'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:02.410440 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1230/2501] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:59:02 2022] DELETE /identity/v3/users/99a660a9790f438fa0ee4d376f7e6707 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:59:02.581631 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:02.583079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:02.597301 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2de7a-d0f2-462c-8d06-3f3b4c9da619 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:02.666329 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:02.667066 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:02.667288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:02.667564 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6 admin admin] PATH_INFO: `/v3/projects/f85a52d1aa3648108739242d9c3f824b` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:02.668758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f85a52d1aa3648108739242d9c3f824b)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:02.670694 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:02.702710 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:59:02.721028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f85a52d1aa3648108739242d9c3f824b', 'request_id': 'req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:02.728559 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f85a52d1aa3648108739242d9c3f824b', 'request_id': 'req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:02.735098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f85a52d1aa3648108739242d9c3f824b', 'request_id': 'req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:02.745780 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6 admin admin] The token cache is being invalidate because project f85a52d1aa3648108739242d9c3f824b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:59:02.746069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:02.746737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:02.748240 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1272/2502] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:59:02 2022] DELETE /identity/v3/projects/f85a52d1aa3648108739242d9c3f824b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:59:04.998131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93c175-bdec-464a-9537-51545e9dbc8c None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:04.998675 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93c175-bdec-464a-9537-51545e9dbc8c None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:04.998858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93c175-bdec-464a-9537-51545e9dbc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:05.021048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-3a93c175-bdec-464a-9537-51545e9dbc8c None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:59:05.022281 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-3a93c175-bdec-464a-9537-51545e9dbc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:05.097559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1231/2503] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 3 12:59:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:59:05.111223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:05.112680 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:05.186015 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba028b10-b6d5-4963-9c1d-b0ee047de8d6 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:05.191952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847339be-0bb1-4930-9491-de9b8fcde401 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:05.192586 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847339be-0bb1-4930-9491-de9b8fcde401 service swift] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:05.192859 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847339be-0bb1-4930-9491-de9b8fcde401 service swift] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:05.193130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847339be-0bb1-4930-9491-de9b8fcde401 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:05.201479 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847339be-0bb1-4930-9491-de9b8fcde401 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:05.203096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847339be-0bb1-4930-9491-de9b8fcde401 service swift] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:05.212187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1273/2504] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:59:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 12:59:15.086202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=46,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:59:15.209044 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:DELETE=6,keystone:SELECT=46,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:59:19.944582 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bba38-f83c-4bde-abca-1fd5c6a8dc9b None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:19.945102 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bba38-f83c-4bde-abca-1fd5c6a8dc9b None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:19.945102 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58bba38-f83c-4bde-abca-1fd5c6a8dc9b None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:19.946565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1232/2505] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 3 12:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:59:19.956871 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29442ea3-dd5f-4c86-9660-1fad7fa39638 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:19.957391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29442ea3-dd5f-4c86-9660-1fad7fa39638 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:19.957640 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29442ea3-dd5f-4c86-9660-1fad7fa39638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:19.994019 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-29442ea3-dd5f-4c86-9660-1fad7fa39638 None None] MFA Rules not processed for user `7b634f567dca42dc95df175a4f36c1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:59:19.995439 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-29442ea3-dd5f-4c86-9660-1fad7fa39638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:20.081470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1274/2506] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 3 12:59:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:59:20.093286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58bba38-f83c-4bde-abca-1fd5c6a8dc9b None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:20.095350 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e58bba38-f83c-4bde-abca-1fd5c6a8dc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:20.167023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58bba38-f83c-4bde-abca-1fd5c6a8dc9b None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:20.170718 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603b257-6c18-44c8-9888-3dc1f079d688 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6066e48aa644a47a889e09cfe204858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:20.171411 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603b257-6c18-44c8-9888-3dc1f079d688 service glance] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:20.172011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603b257-6c18-44c8-9888-3dc1f079d688 service glance] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:20.172350 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603b257-6c18-44c8-9888-3dc1f079d688 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:20.183468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603b257-6c18-44c8-9888-3dc1f079d688 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:20.184885 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1603b257-6c18-44c8-9888-3dc1f079d688 service glance] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:20.196279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1233/2507] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:59:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:59:20.267769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9b45b-2d74-4eee-850a-634411e7b2d3 None None] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:20.268062 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9b45b-2d74-4eee-850a-634411e7b2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:20.268339 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9b45b-2d74-4eee-850a-634411e7b2d3 None None] PATH_INFO: `/` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:20.269681 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1275/2508] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 3 12:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:59:20.275138 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54d6d0-6b2d-40dc-9fb3-91bb858093f3 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:20.275510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54d6d0-6b2d-40dc-9fb3-91bb858093f3 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:20.275788 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe54d6d0-6b2d-40dc-9fb3-91bb858093f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:20.296033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-fe54d6d0-6b2d-40dc-9fb3-91bb858093f3 None None] MFA Rules not processed for user `52ff458fd7f74b99a745edbcf13a114b`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:59:20.297564 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-fe54d6d0-6b2d-40dc-9fb3-91bb858093f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:20.349650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1234/2509] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 3 12:59:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:59:20.361277 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9b45b-2d74-4eee-850a-634411e7b2d3 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:20.362953 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-45a9b45b-2d74-4eee-850a-634411e7b2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:20.375706 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9b45b-2d74-4eee-850a-634411e7b2d3 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:20.379185 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69520f8-8f19-4df4-b15d-41932aea2351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:20.379778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69520f8-8f19-4df4-b15d-41932aea2351 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:20.380053 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69520f8-8f19-4df4-b15d-41932aea2351 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:20.380324 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69520f8-8f19-4df4-b15d-41932aea2351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:20.387127 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69520f8-8f19-4df4-b15d-41932aea2351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:20.388518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69520f8-8f19-4df4-b15d-41932aea2351 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:20.396565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1276/2510] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:59:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:59:20.477486 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d378f36-664f-4e78-b53a-2308efc387ad None None] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:20.477779 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d378f36-664f-4e78-b53a-2308efc387ad None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:20.478055 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d378f36-664f-4e78-b53a-2308efc387ad None None] PATH_INFO: `/` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:20.479495 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1235/2511] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 3 12:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 03 12:59:20.485085 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbec79-a083-465e-8c66-2281173a64eb None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:20.485370 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbec79-a083-465e-8c66-2281173a64eb None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:20.486286 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbec79-a083-465e-8c66-2281173a64eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:20.505359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-d5fbec79-a083-465e-8c66-2281173a64eb None None] MFA Rules not processed for user `7b634f567dca42dc95df175a4f36c1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:59:20.506586 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d5fbec79-a083-465e-8c66-2281173a64eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:20.515757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1277/2512] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 3 12:59:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:59:20.526594 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d378f36-664f-4e78-b53a-2308efc387ad None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:20.538169 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d378f36-664f-4e78-b53a-2308efc387ad None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:20.541718 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e260b6-69c0-42f9-bd1a-184b2d7e5e9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:20.542323 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e260b6-69c0-42f9-bd1a-184b2d7e5e9e service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:20.542648 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e260b6-69c0-42f9-bd1a-184b2d7e5e9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:20.542921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e260b6-69c0-42f9-bd1a-184b2d7e5e9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:20.553296 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e260b6-69c0-42f9-bd1a-184b2d7e5e9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:20.554731 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e260b6-69c0-42f9-bd1a-184b2d7e5e9e service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:20.565810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1236/2513] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:59:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:59:23.333577 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109a56f-350a-443e-bc6d-acf4f17c30f5 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:23.334086 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109a56f-350a-443e-bc6d-acf4f17c30f5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:23.334191 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109a56f-350a-443e-bc6d-acf4f17c30f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:23.352849 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-5109a56f-350a-443e-bc6d-acf4f17c30f5 None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:59:23.354068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5109a56f-350a-443e-bc6d-acf4f17c30f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:23.361874 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1278/2514] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 3 12:59:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:59:23.375726 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e260b6-69c0-42f9-bd1a-184b2d7e5e9e service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:23.388062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e260b6-69c0-42f9-bd1a-184b2d7e5e9e service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:23.393139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3c4f8-b4fa-420e-a879-bc662cb30eae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:23.393845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3c4f8-b4fa-420e-a879-bc662cb30eae service swift] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:23.394198 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3c4f8-b4fa-420e-a879-bc662cb30eae service swift] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:23.394587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f3c4f8-b4fa-420e-a879-bc662cb30eae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:23.403196 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3c4f8-b4fa-420e-a879-bc662cb30eae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:23.404855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f3c4f8-b4fa-420e-a879-bc662cb30eae service swift] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:23.413134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1237/2515] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:59:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 12:59:33.348067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=43 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:59:33.409732 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=61 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:59:42.469110 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5109a56f-350a-443e-bc6d-acf4f17c30f5 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:42.471052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5109a56f-350a-443e-bc6d-acf4f17c30f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:42.483633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5109a56f-350a-443e-bc6d-acf4f17c30f5 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:42.487966 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e9799-a705-40ac-ad69-65176408bee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:42.489033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e9799-a705-40ac-ad69-65176408bee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:42.489318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e9799-a705-40ac-ad69-65176408bee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:42.490016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e9799-a705-40ac-ad69-65176408bee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:42.491872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-436e9799-a705-40ac-ad69-65176408bee5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:42.555701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e9799-a705-40ac-ad69-65176408bee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:42.557159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e9799-a705-40ac-ad69-65176408bee5 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:42.567202 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1279/2516] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:59:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:59:45.868315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3c4f8-b4fa-420e-a879-bc662cb30eae service swift] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:45.869785 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f3f3c4f8-b4fa-420e-a879-bc662cb30eae service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:45.884902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f3c4f8-b4fa-420e-a879-bc662cb30eae service swift] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:45.889636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4511fc-bbcb-4577-ba7d-50f918d5700d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:45.890278 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4511fc-bbcb-4577-ba7d-50f918d5700d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:45.890624 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4511fc-bbcb-4577-ba7d-50f918d5700d admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:45.890903 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4511fc-bbcb-4577-ba7d-50f918d5700d admin admin] PATH_INFO: `/v3/users/5f16ff28aac54514a475f8a4f5c6e4c4` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:45.892597 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4511fc-bbcb-4577-ba7d-50f918d5700d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f16ff28aac54514a475f8a4f5c6e4c4)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:45.894555 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4511fc-bbcb-4577-ba7d-50f918d5700d admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:45.941650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9a4511fc-bbcb-4577-ba7d-50f918d5700d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f16ff28aac54514a475f8a4f5c6e4c4'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:45.953477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9a4511fc-bbcb-4577-ba7d-50f918d5700d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f16ff28aac54514a475f8a4f5c6e4c4'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:45.961400 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9a4511fc-bbcb-4577-ba7d-50f918d5700d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f16ff28aac54514a475f8a4f5c6e4c4'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:45.962188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9a4511fc-bbcb-4577-ba7d-50f918d5700d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f16ff28aac54514a475f8a4f5c6e4c4'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:45.969056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9a4511fc-bbcb-4577-ba7d-50f918d5700d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f16ff28aac54514a475f8a4f5c6e4c4'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:45.975531 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1238/2517] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 12:59:45 2022] DELETE /identity/v3/users/5f16ff28aac54514a475f8a4f5c6e4c4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:59:46.146660 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e9799-a705-40ac-ad69-65176408bee5 service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:46.156459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e9799-a705-40ac-ad69-65176408bee5 service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:46.214575 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5867a4a-3294-4e69-8902-624f2a94230f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:46.215211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5867a4a-3294-4e69-8902-624f2a94230f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:46.215487 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5867a4a-3294-4e69-8902-624f2a94230f admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:46.215756 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5867a4a-3294-4e69-8902-624f2a94230f admin admin] PATH_INFO: `/v3/projects/d5e359c6007d4781972a3c09d817e2bd` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:46.216910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5867a4a-3294-4e69-8902-624f2a94230f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5e359c6007d4781972a3c09d817e2bd)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:46.218831 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5867a4a-3294-4e69-8902-624f2a94230f admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:46.262910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c5867a4a-3294-4e69-8902-624f2a94230f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5e359c6007d4781972a3c09d817e2bd', 'request_id': 'req-c5867a4a-3294-4e69-8902-624f2a94230f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:46.270667 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c5867a4a-3294-4e69-8902-624f2a94230f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5e359c6007d4781972a3c09d817e2bd', 'request_id': 'req-c5867a4a-3294-4e69-8902-624f2a94230f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:46.276544 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c5867a4a-3294-4e69-8902-624f2a94230f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5e359c6007d4781972a3c09d817e2bd', 'request_id': 'req-c5867a4a-3294-4e69-8902-624f2a94230f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:46.287977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c5867a4a-3294-4e69-8902-624f2a94230f admin admin] The token cache is being invalidate because project d5e359c6007d4781972a3c09d817e2bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 12:59:46.288264 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c5867a4a-3294-4e69-8902-624f2a94230f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:46.288891 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c5867a4a-3294-4e69-8902-624f2a94230f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 12:59:46.290385 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1280/2518] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 12:59:46 2022] DELETE /identity/v3/projects/d5e359c6007d4781972a3c09d817e2bd => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:59:46.295645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fc2ad-0860-4236-ac74-a0cf263beb3a None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:46.296785 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fc2ad-0860-4236-ac74-a0cf263beb3a None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:46.297249 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9fc2ad-0860-4236-ac74-a0cf263beb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:46.317987 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-7d9fc2ad-0860-4236-ac74-a0cf263beb3a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:59:46.319225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-7d9fc2ad-0860-4236-ac74-a0cf263beb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:46.377570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1239/2519] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:59:46.385336 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf62d3-3d82-46dc-a597-cf91633ead98 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:46.385700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf62d3-3d82-46dc-a597-cf91633ead98 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:46.385981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf62d3-3d82-46dc-a597-cf91633ead98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:46.407570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-4aaf62d3-3d82-46dc-a597-cf91633ead98 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:59:46.408778 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-4aaf62d3-3d82-46dc-a597-cf91633ead98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:46.416882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1281/2520] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:59:46.421076 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fc2ad-0860-4236-ac74-a0cf263beb3a None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:46.431104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9fc2ad-0860-4236-ac74-a0cf263beb3a None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:46.435638 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5c001-7030-4d0f-940c-3fddbdb24b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:46.436244 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5c001-7030-4d0f-940c-3fddbdb24b79 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:46.436527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5c001-7030-4d0f-940c-3fddbdb24b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:46.436801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5c001-7030-4d0f-940c-3fddbdb24b79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:46.437498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5c001-7030-4d0f-940c-3fddbdb24b79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 12:59:46.437800 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5c001-7030-4d0f-940c-3fddbdb24b79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:46.439947 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5c001-7030-4d0f-940c-3fddbdb24b79 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:46.456291 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1240/2521] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 12:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 12:59:46.457343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf62d3-3d82-46dc-a597-cf91633ead98 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:46.467900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf62d3-3d82-46dc-a597-cf91633ead98 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:46.473518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3cee7d-84eb-402c-bcaf-b536ad493447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:46.474176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cee7d-84eb-402c-bcaf-b536ad493447 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:46.474450 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cee7d-84eb-402c-bcaf-b536ad493447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:46.474747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3cee7d-84eb-402c-bcaf-b536ad493447 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:46.475411 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3cee7d-84eb-402c-bcaf-b536ad493447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:46.477668 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3cee7d-84eb-402c-bcaf-b536ad493447 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:46.497621 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:59:46.514533 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1282/2522] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 12:59:46 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:59:46.519068 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5c001-7030-4d0f-940c-3fddbdb24b79 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:46.530095 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5c001-7030-4d0f-940c-3fddbdb24b79 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:46.536933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227454bc-7ec7-434e-bc6e-ab21da30e418 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:46.537215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227454bc-7ec7-434e-bc6e-ab21da30e418 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:46.537481 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227454bc-7ec7-434e-bc6e-ab21da30e418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:46.559722 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-227454bc-7ec7-434e-bc6e-ab21da30e418 None None] MFA Rules not processed for user `3ae6eeb3d4f94925a5e1dd497d352d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:59:46.560930 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-227454bc-7ec7-434e-bc6e-ab21da30e418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:46.593863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e424dfb-e262-4a62-9aa1-aea6faf49f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:46.594509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e424dfb-e262-4a62-9aa1-aea6faf49f26 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:46.594807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e424dfb-e262-4a62-9aa1-aea6faf49f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:46.595715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e424dfb-e262-4a62-9aa1-aea6faf49f26 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:46.596373 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e424dfb-e262-4a62-9aa1-aea6faf49f26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:46.598274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e424dfb-e262-4a62-9aa1-aea6faf49f26 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:46.606885 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:59:46.631354 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1241/2523] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 12:59:46 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 12:59:46.632493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e424dfb-e262-4a62-9aa1-aea6faf49f26 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:46.633328 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1283/2524] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 3 12:59:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:59:46.642888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e424dfb-e262-4a62-9aa1-aea6faf49f26 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:46.646246 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227454bc-7ec7-434e-bc6e-ab21da30e418 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:46.647669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-227454bc-7ec7-434e-bc6e-ab21da30e418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:46.648027 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fc6e7-d8bc-478b-8657-37844dc2838a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:46.648642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fc6e7-d8bc-478b-8657-37844dc2838a admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:46.648922 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fc6e7-d8bc-478b-8657-37844dc2838a admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:46.649203 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fc6e7-d8bc-478b-8657-37844dc2838a admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:46.649899 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fc6e7-d8bc-478b-8657-37844dc2838a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:46.651530 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fc6e7-d8bc-478b-8657-37844dc2838a admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:46.663198 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1242/2525] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 12:59:46.668542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fc6e7-d8bc-478b-8657-37844dc2838a admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:46.679322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fc6e7-d8bc-478b-8657-37844dc2838a admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:46.685014 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddffb5c-ad49-46ad-ab61-367468b9be51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:46.685645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddffb5c-ad49-46ad-ab61-367468b9be51 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:46.685921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddffb5c-ad49-46ad-ab61-367468b9be51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:46.686190 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddffb5c-ad49-46ad-ab61-367468b9be51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:46.686912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddffb5c-ad49-46ad-ab61-367468b9be51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:46.688573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddffb5c-ad49-46ad-ab61-367468b9be51 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:46.701725 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1243/2526] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 12:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 12:59:46.702827 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddffb5c-ad49-46ad-ab61-367468b9be51 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:46.712896 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddffb5c-ad49-46ad-ab61-367468b9be51 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:46.717636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03db3b-377b-45c4-b34a-7a475239c5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:46.718359 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03db3b-377b-45c4-b34a-7a475239c5de admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:46.718763 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03db3b-377b-45c4-b34a-7a475239c5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:46.719005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227454bc-7ec7-434e-bc6e-ab21da30e418 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:46.719113 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed03db3b-377b-45c4-b34a-7a475239c5de admin admin] PATH_INFO: `/v3/projects/8ed92e8abb0c4adf85c6854934423f1e/users/fd489ef668494ec98cf9d167a1ded3ce/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:46.724899 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7faf6-583d-4a59-be6b-b66b1af99770 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cea54cb772344612aec4686d484de89f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:46.725630 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7faf6-583d-4a59-be6b-b66b1af99770 service swift] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:46.726066 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7faf6-583d-4a59-be6b-b66b1af99770 service swift] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:46.726364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7faf6-583d-4a59-be6b-b66b1af99770 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:46.735059 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7faf6-583d-4a59-be6b-b66b1af99770 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:46.736119 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03db3b-377b-45c4-b34a-7a475239c5de admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ed92e8abb0c4adf85c6854934423f1e, user_id=fd489ef668494ec98cf9d167a1ded3ce, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:46.737231 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7faf6-583d-4a59-be6b-b66b1af99770 service swift] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:46.738825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed03db3b-377b-45c4-b34a-7a475239c5de admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:46.745475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1284/2527] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 3 12:59:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 03 12:59:46.752771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1244/2528] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 12:59:46 2022] PUT /identity/v3/projects/8ed92e8abb0c4adf85c6854934423f1e/users/fd489ef668494ec98cf9d167a1ded3ce/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 12:59:46.758230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ddcc2-add7-48dd-a413-3b1f3ea19b8d None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:46.758537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ddcc2-add7-48dd-a413-3b1f3ea19b8d None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:46.758813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ddcc2-add7-48dd-a413-3b1f3ea19b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:46.778465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-db7ddcc2-add7-48dd-a413-3b1f3ea19b8d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:59:46.779849 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-db7ddcc2-add7-48dd-a413-3b1f3ea19b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:46.841931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1285/2529] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 12:59:46.852522 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03db3b-377b-45c4-b34a-7a475239c5de admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:46.854199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ed03db3b-377b-45c4-b34a-7a475239c5de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:46.916678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed03db3b-377b-45c4-b34a-7a475239c5de admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:46.920639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f38073-cf53-4612-8243-afa610928061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:46.921522 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f38073-cf53-4612-8243-afa610928061 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:46.921804 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f38073-cf53-4612-8243-afa610928061 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:46.922082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f38073-cf53-4612-8243-afa610928061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:46.932015 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f38073-cf53-4612-8243-afa610928061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:46.933445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f38073-cf53-4612-8243-afa610928061 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:46.950681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1245/2530] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:59:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:59:51.080915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459a7f7-4f2e-43a6-aea5-5e44258af615 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:51.081398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459a7f7-4f2e-43a6-aea5-5e44258af615 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:51.081497 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459a7f7-4f2e-43a6-aea5-5e44258af615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:51.133429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-a459a7f7-4f2e-43a6-aea5-5e44258af615 None None] MFA Rules not processed for user `fd489ef668494ec98cf9d167a1ded3ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:59:51.136275 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a459a7f7-4f2e-43a6-aea5-5e44258af615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:51.174939 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 12:59:51.248505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1286/2531] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:59:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 12:59:51.258111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f38073-cf53-4612-8243-afa610928061 service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 12:59:51.259744 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-01f38073-cf53-4612-8243-afa610928061 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:51.352344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f38073-cf53-4612-8243-afa610928061 service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 12:59:51.358336 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a52faa-f684-4146-833e-c1be5612c2c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 12:59:51.359012 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a52faa-f684-4146-833e-c1be5612c2c2 service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:51.359308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a52faa-f684-4146-833e-c1be5612c2c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:51.359585 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a52faa-f684-4146-833e-c1be5612c2c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:51.368445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a52faa-f684-4146-833e-c1be5612c2c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 12:59:51.370271 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a52faa-f684-4146-833e-c1be5612c2c2 service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 12:59:51.381988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1246/2532] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 12:59:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 12:59:52.782413 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b264403-dfeb-4267-984a-1c7ce4fc6232 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 12:59:52.782948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b264403-dfeb-4267-984a-1c7ce4fc6232 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 12:59:52.783050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b264403-dfeb-4267-984a-1c7ce4fc6232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 12:59:52.803269 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-2b264403-dfeb-4267-984a-1c7ce4fc6232 None None] MFA Rules not processed for user `fd489ef668494ec98cf9d167a1ded3ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 12:59:52.804472 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-2b264403-dfeb-4267-984a-1c7ce4fc6232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 12:59:52.811293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1287/2533] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 12:59:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:00:01.375307 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:INSERT=4,keystone:SELECT=82 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:00:02.797947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=21 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:00:03.814427 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c35192-89dc-4d7b-b7b2-752a365a21c4 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:03.815451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c35192-89dc-4d7b-b7b2-752a365a21c4 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:03.815755 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c35192-89dc-4d7b-b7b2-752a365a21c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:03.835862 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-81c35192-89dc-4d7b-b7b2-752a365a21c4 None None] MFA Rules not processed for user `6d333c7a05af409f811c4382f048d1fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:00:03.837120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-81c35192-89dc-4d7b-b7b2-752a365a21c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:03.897949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1247/2534] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:00:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:00:13.890655 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=25 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:00:33.143416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b264403-dfeb-4267-984a-1c7ce4fc6232 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:33.144852 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-2b264403-dfeb-4267-984a-1c7ce4fc6232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:33.199910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b264403-dfeb-4267-984a-1c7ce4fc6232 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:33.203410 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f40b8-31d0-4829-abb4-18522a4008e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:33.204029 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f40b8-31d0-4829-abb4-18522a4008e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:33.204310 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f40b8-31d0-4829-abb4-18522a4008e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:33.204576 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f40b8-31d0-4829-abb4-18522a4008e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:33.206264 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-887f40b8-31d0-4829-abb4-18522a4008e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:33.262327 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f40b8-31d0-4829-abb4-18522a4008e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:33.263773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f40b8-31d0-4829-abb4-18522a4008e2 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:33.272774 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1288/2535] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:00:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:00:33.383356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c35192-89dc-4d7b-b7b2-752a365a21c4 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:33.393266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c35192-89dc-4d7b-b7b2-752a365a21c4 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:33.396798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2aa90b-fd11-4dd3-b93c-a093ecb75f91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:33.397454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2aa90b-fd11-4dd3-b93c-a093ecb75f91 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:33.397734 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2aa90b-fd11-4dd3-b93c-a093ecb75f91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:33.398008 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2aa90b-fd11-4dd3-b93c-a093ecb75f91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:33.399867 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8a2aa90b-fd11-4dd3-b93c-a093ecb75f91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:33.413136 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2aa90b-fd11-4dd3-b93c-a093ecb75f91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:33.414638 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2aa90b-fd11-4dd3-b93c-a093ecb75f91 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:33.426532 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1248/2536] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:00:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:00:36.458951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f40b8-31d0-4829-abb4-18522a4008e2 service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:36.460470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-887f40b8-31d0-4829-abb4-18522a4008e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:36.517131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2aa90b-fd11-4dd3-b93c-a093ecb75f91 service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:36.518859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8a2aa90b-fd11-4dd3-b93c-a093ecb75f91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:36.540621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2aa90b-fd11-4dd3-b93c-a093ecb75f91 service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:36.544832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f40b8-31d0-4829-abb4-18522a4008e2 service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:36.547237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a9462-c0e9-4b4d-be3b-a9fef8dcbb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:36.548687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a9462-c0e9-4b4d-be3b-a9fef8dcbb1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:36.549298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a9462-c0e9-4b4d-be3b-a9fef8dcbb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:36.549798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a9462-c0e9-4b4d-be3b-a9fef8dcbb1c admin admin] PATH_INFO: `/v3/users/13128464bb2547f5838a787d022e6c5f` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:36.551621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a9462-c0e9-4b4d-be3b-a9fef8dcbb1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=13128464bb2547f5838a787d022e6c5f)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:36.553718 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6a9462-c0e9-4b4d-be3b-a9fef8dcbb1c admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:36.554248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827da32-8d11-4297-af26-b0a4b7364774 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5094263ac471481db48783b19192e01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:36.554919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827da32-8d11-4297-af26-b0a4b7364774 service placement] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:36.555247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827da32-8d11-4297-af26-b0a4b7364774 service placement] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:36.555574 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827da32-8d11-4297-af26-b0a4b7364774 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:36.562727 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0827da32-8d11-4297-af26-b0a4b7364774 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:36.574642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827da32-8d11-4297-af26-b0a4b7364774 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:36.576202 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827da32-8d11-4297-af26-b0a4b7364774 service placement] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:36.587327 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1289/2537] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:00:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:00:36.607363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-da6a9462-c0e9-4b4d-be3b-a9fef8dcbb1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13128464bb2547f5838a787d022e6c5f'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:36.622943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-da6a9462-c0e9-4b4d-be3b-a9fef8dcbb1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13128464bb2547f5838a787d022e6c5f'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:36.630880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-da6a9462-c0e9-4b4d-be3b-a9fef8dcbb1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13128464bb2547f5838a787d022e6c5f'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:36.633467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-da6a9462-c0e9-4b4d-be3b-a9fef8dcbb1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13128464bb2547f5838a787d022e6c5f'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:36.640722 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-da6a9462-c0e9-4b4d-be3b-a9fef8dcbb1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13128464bb2547f5838a787d022e6c5f'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:36.646841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1249/2538] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:00:36 2022] DELETE /identity/v3/users/13128464bb2547f5838a787d022e6c5f => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 13:00:36.819285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827da32-8d11-4297-af26-b0a4b7364774 service placement] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:36.831668 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827da32-8d11-4297-af26-b0a4b7364774 service placement] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:36.873899 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:00:36.888987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cba4f-21ce-4bcd-8c75-bdb752000418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:36.889623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cba4f-21ce-4bcd-8c75-bdb752000418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:36.889901 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cba4f-21ce-4bcd-8c75-bdb752000418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:36.890171 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cba4f-21ce-4bcd-8c75-bdb752000418 admin admin] PATH_INFO: `/v3/projects/1608d8997d504543a9cae8e8307581d5` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:36.891367 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cba4f-21ce-4bcd-8c75-bdb752000418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1608d8997d504543a9cae8e8307581d5)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:36.893722 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cba4f-21ce-4bcd-8c75-bdb752000418 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:36.938247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a91cba4f-21ce-4bcd-8c75-bdb752000418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1608d8997d504543a9cae8e8307581d5', 'request_id': 'req-a91cba4f-21ce-4bcd-8c75-bdb752000418'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:36.945298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a91cba4f-21ce-4bcd-8c75-bdb752000418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1608d8997d504543a9cae8e8307581d5', 'request_id': 'req-a91cba4f-21ce-4bcd-8c75-bdb752000418'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:36.951589 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a91cba4f-21ce-4bcd-8c75-bdb752000418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1608d8997d504543a9cae8e8307581d5', 'request_id': 'req-a91cba4f-21ce-4bcd-8c75-bdb752000418'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:36.962956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a91cba4f-21ce-4bcd-8c75-bdb752000418 admin admin] The token cache is being invalidate because project 1608d8997d504543a9cae8e8307581d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:00:36.963241 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a91cba4f-21ce-4bcd-8c75-bdb752000418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:36.963881 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a91cba4f-21ce-4bcd-8c75-bdb752000418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:36.965401 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1290/2539] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:00:36 2022] DELETE /identity/v3/projects/1608d8997d504543a9cae8e8307581d5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:00:36.970849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ad19d-232e-4fd0-b6ca-770e24607256 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:36.971027 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ad19d-232e-4fd0-b6ca-770e24607256 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:36.971326 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ad19d-232e-4fd0-b6ca-770e24607256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:36.992428 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-f76ad19d-232e-4fd0-b6ca-770e24607256 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:00:36.993599 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f76ad19d-232e-4fd0-b6ca-770e24607256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:37.069286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1250/2540] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 13:00:37.077174 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160f996-cf7a-4e81-9084-c0b23350466e None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:37.077529 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160f996-cf7a-4e81-9084-c0b23350466e None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:37.077829 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160f996-cf7a-4e81-9084-c0b23350466e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:37.097699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-f160f996-cf7a-4e81-9084-c0b23350466e None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:00:37.099013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f160f996-cf7a-4e81-9084-c0b23350466e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:37.107008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1291/2541] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:00:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:00:37.111601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ad19d-232e-4fd0-b6ca-770e24607256 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:37.121415 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ad19d-232e-4fd0-b6ca-770e24607256 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:37.126603 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cfe1b-4942-4a14-a2b4-38228f12afed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:37.127216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cfe1b-4942-4a14-a2b4-38228f12afed admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:37.127572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cfe1b-4942-4a14-a2b4-38228f12afed admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:37.127854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cfe1b-4942-4a14-a2b4-38228f12afed admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:37.128546 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cfe1b-4942-4a14-a2b4-38228f12afed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 13:00:37.128845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cfe1b-4942-4a14-a2b4-38228f12afed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:37.130550 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386cfe1b-4942-4a14-a2b4-38228f12afed admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:37.142886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1251/2542] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 13:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 13:00:37.147289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160f996-cf7a-4e81-9084-c0b23350466e None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:37.156893 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160f996-cf7a-4e81-9084-c0b23350466e None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:37.161753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa681a0-4cec-4658-86f6-bca963506de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:37.162384 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa681a0-4cec-4658-86f6-bca963506de4 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:37.162815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa681a0-4cec-4658-86f6-bca963506de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:37.163087 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa681a0-4cec-4658-86f6-bca963506de4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:37.163762 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa681a0-4cec-4658-86f6-bca963506de4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:37.165677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa681a0-4cec-4658-86f6-bca963506de4 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:37.193464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1292/2543] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:00:37 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 13:00:37.197638 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cfe1b-4942-4a14-a2b4-38228f12afed admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:37.208880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cfe1b-4942-4a14-a2b4-38228f12afed admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:37.254499 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:00:37.268883 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa7b47-8b05-4413-bb53-1e06d075b981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:37.269489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa7b47-8b05-4413-bb53-1e06d075b981 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:37.269767 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa7b47-8b05-4413-bb53-1e06d075b981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:37.270038 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa7b47-8b05-4413-bb53-1e06d075b981 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:37.270729 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa7b47-8b05-4413-bb53-1e06d075b981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:37.272616 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa7b47-8b05-4413-bb53-1e06d075b981 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:37.293320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1252/2544] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:00:37 2022] POST /identity/v3/users => generated 279 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 13:00:37.297784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa681a0-4cec-4658-86f6-bca963506de4 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:37.308184 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa681a0-4cec-4658-86f6-bca963506de4 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:37.312926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc40df5-0ec8-4d38-97fb-8744724e773c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:37.314026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc40df5-0ec8-4d38-97fb-8744724e773c admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:37.314308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc40df5-0ec8-4d38-97fb-8744724e773c admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:37.314619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc40df5-0ec8-4d38-97fb-8744724e773c admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:37.315311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc40df5-0ec8-4d38-97fb-8744724e773c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:37.316932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc40df5-0ec8-4d38-97fb-8744724e773c admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:37.327293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1293/2545] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:00:37.331635 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa7b47-8b05-4413-bb53-1e06d075b981 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:37.342570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa7b47-8b05-4413-bb53-1e06d075b981 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:37.347535 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a8ace8-388a-414e-8376-f2f62021151d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:37.348144 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8ace8-388a-414e-8376-f2f62021151d admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:37.348427 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8ace8-388a-414e-8376-f2f62021151d admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:37.348704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a8ace8-388a-414e-8376-f2f62021151d admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:37.349397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8ace8-388a-414e-8376-f2f62021151d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:37.351058 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a8ace8-388a-414e-8376-f2f62021151d admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:37.361581 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1253/2546] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:00:37.365861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc40df5-0ec8-4d38-97fb-8744724e773c admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:37.376359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc40df5-0ec8-4d38-97fb-8744724e773c admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:37.380898 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d13d3-b9c9-4682-862a-23c60ff33acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:37.381547 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d13d3-b9c9-4682-862a-23c60ff33acb admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:37.381826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d13d3-b9c9-4682-862a-23c60ff33acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:37.382099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d13d3-b9c9-4682-862a-23c60ff33acb admin admin] PATH_INFO: `/v3/projects/593071d79dca40eca7fc581f74f570ca/users/86d7018443694536a7a70f46e59ff93d/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:37.397023 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d13d3-b9c9-4682-862a-23c60ff33acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=593071d79dca40eca7fc581f74f570ca, user_id=86d7018443694536a7a70f46e59ff93d, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:37.399690 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d13d3-b9c9-4682-862a-23c60ff33acb admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:37.412572 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1294/2547] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:00:37 2022] PUT /identity/v3/projects/593071d79dca40eca7fc581f74f570ca/users/86d7018443694536a7a70f46e59ff93d/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:00:37.423621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5553209-9dff-481f-a293-dff63dfe900e None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:37.423910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5553209-9dff-481f-a293-dff63dfe900e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:37.424187 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5553209-9dff-481f-a293-dff63dfe900e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:37.451343 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-e5553209-9dff-481f-a293-dff63dfe900e None None] MFA Rules not processed for user `86d7018443694536a7a70f46e59ff93d`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:00:37.452549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e5553209-9dff-481f-a293-dff63dfe900e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:37.515584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1254/2548] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:00:37 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:00:37.520158 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d13d3-b9c9-4682-862a-23c60ff33acb admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:37.582931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d13d3-b9c9-4682-862a-23c60ff33acb admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:37.587931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1611de-9e68-4c1f-899a-2728906387ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:37.588539 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1611de-9e68-4c1f-899a-2728906387ce admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:37.588817 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1611de-9e68-4c1f-899a-2728906387ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:37.589088 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1611de-9e68-4c1f-899a-2728906387ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:37.589757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1611de-9e68-4c1f-899a-2728906387ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:37.591669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1611de-9e68-4c1f-899a-2728906387ce admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:37.607347 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:00:37.622207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1295/2549] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:00:37 2022] POST /identity/v3/projects => generated 357 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:00:37.627099 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5553209-9dff-481f-a293-dff63dfe900e None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:37.637925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5553209-9dff-481f-a293-dff63dfe900e None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:37.698634 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099d52e-2f09-4c56-a64c-32452d7c7aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:37.699255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099d52e-2f09-4c56-a64c-32452d7c7aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:37.699544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099d52e-2f09-4c56-a64c-32452d7c7aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:37.699811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099d52e-2f09-4c56-a64c-32452d7c7aa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:37.700476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099d52e-2f09-4c56-a64c-32452d7c7aa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:37.702390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9099d52e-2f09-4c56-a64c-32452d7c7aa3 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:37.725035 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1255/2550] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:00:37 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:00:37.728984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1611de-9e68-4c1f-899a-2728906387ce admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:37.739794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1611de-9e68-4c1f-899a-2728906387ce admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:37.744519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdd260-437e-4437-86d4-829c6c56df38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:37.745187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdd260-437e-4437-86d4-829c6c56df38 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:37.745531 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdd260-437e-4437-86d4-829c6c56df38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:37.745893 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdd260-437e-4437-86d4-829c6c56df38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:37.746684 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdd260-437e-4437-86d4-829c6c56df38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:37.748326 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdd260-437e-4437-86d4-829c6c56df38 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:37.762018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1296/2551] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 13:00:37.766267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099d52e-2f09-4c56-a64c-32452d7c7aa3 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:37.778270 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099d52e-2f09-4c56-a64c-32452d7c7aa3 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:37.783588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94ec1c-6944-41d9-a8ca-87237c5245a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:37.784248 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94ec1c-6944-41d9-a8ca-87237c5245a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:37.784529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94ec1c-6944-41d9-a8ca-87237c5245a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:37.784807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94ec1c-6944-41d9-a8ca-87237c5245a0 admin admin] PATH_INFO: `/v3/projects/b72cbb4795634380a7c9e50c71763492/users/ae0692d6a427482f9fb907e12442b525/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:37.800153 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94ec1c-6944-41d9-a8ca-87237c5245a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b72cbb4795634380a7c9e50c71763492, user_id=ae0692d6a427482f9fb907e12442b525, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:37.802873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94ec1c-6944-41d9-a8ca-87237c5245a0 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:37.816280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1256/2552] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:00:37 2022] PUT /identity/v3/projects/b72cbb4795634380a7c9e50c71763492/users/ae0692d6a427482f9fb907e12442b525/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:00:37.820538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdd260-437e-4437-86d4-829c6c56df38 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:37.830828 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdd260-437e-4437-86d4-829c6c56df38 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:37.896196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41a54b-9c21-4dbc-8f1c-114704299f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:37.896832 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41a54b-9c21-4dbc-8f1c-114704299f6e admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:37.897148 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41a54b-9c21-4dbc-8f1c-114704299f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:37.897566 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41a54b-9c21-4dbc-8f1c-114704299f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:37.898292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41a54b-9c21-4dbc-8f1c-114704299f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:37.900002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41a54b-9c21-4dbc-8f1c-114704299f6e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:37.916045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1297/2553] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:00:37.922716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94ec1c-6944-41d9-a8ca-87237c5245a0 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:37.935966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94ec1c-6944-41d9-a8ca-87237c5245a0 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:37.940711 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563e120-a388-4043-a066-f50ec5f1b2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:37.941373 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563e120-a388-4043-a066-f50ec5f1b2d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:37.941651 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563e120-a388-4043-a066-f50ec5f1b2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:37.941924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563e120-a388-4043-a066-f50ec5f1b2d3 admin admin] PATH_INFO: `/v3/projects/b72cbb4795634380a7c9e50c71763492/users/ae0692d6a427482f9fb907e12442b525/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:37.943995 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4563e120-a388-4043-a066-f50ec5f1b2d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b72cbb4795634380a7c9e50c71763492, user_id=ae0692d6a427482f9fb907e12442b525, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:37.947054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4563e120-a388-4043-a066-f50ec5f1b2d3 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:37.959797 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1257/2554] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:00:37 2022] PUT /identity/v3/projects/b72cbb4795634380a7c9e50c71763492/users/ae0692d6a427482f9fb907e12442b525/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:00:37.977248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4edcf-b9b9-43a6-914f-614252fa00aa None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:37.977648 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4edcf-b9b9-43a6-914f-614252fa00aa None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:37.978027 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4edcf-b9b9-43a6-914f-614252fa00aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:38.008757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-bce4edcf-b9b9-43a6-914f-614252fa00aa None None] MFA Rules not processed for user `ae0692d6a427482f9fb907e12442b525`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:00:38.009965 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-bce4edcf-b9b9-43a6-914f-614252fa00aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:38.080385 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1298/2555] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:00:37 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:00:38.091352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563e120-a388-4043-a066-f50ec5f1b2d3 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:38.093064 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-4563e120-a388-4043-a066-f50ec5f1b2d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:38.103445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:00:38.152871 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563e120-a388-4043-a066-f50ec5f1b2d3 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:38.156554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41bd3e-f47f-4562-9ed9-2f5c2d4f82b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:38.157669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41bd3e-f47f-4562-9ed9-2f5c2d4f82b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:38.157946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41bd3e-f47f-4562-9ed9-2f5c2d4f82b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:38.158214 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41bd3e-f47f-4562-9ed9-2f5c2d4f82b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:38.167475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41bd3e-f47f-4562-9ed9-2f5c2d4f82b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:38.168883 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e41bd3e-f47f-4562-9ed9-2f5c2d4f82b6 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:38.179174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1258/2556] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:00:38 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:00:38.257720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4edcf-b9b9-43a6-914f-614252fa00aa None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:38.259577 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-bce4edcf-b9b9-43a6-914f-614252fa00aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:38.274105 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4edcf-b9b9-43a6-914f-614252fa00aa None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:38.277741 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da1732-a715-447d-8515-b49eebcb8113 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:38.278383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da1732-a715-447d-8515-b49eebcb8113 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:38.278691 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da1732-a715-447d-8515-b49eebcb8113 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:38.278963 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da1732-a715-447d-8515-b49eebcb8113 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:38.342458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da1732-a715-447d-8515-b49eebcb8113 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:38.344114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da1732-a715-447d-8515-b49eebcb8113 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:38.354421 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1299/2557] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:00:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:00:41.194239 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365ff9e-57a0-4c76-b9f3-503d1561d537 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:41.194754 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365ff9e-57a0-4c76-b9f3-503d1561d537 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:41.194754 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365ff9e-57a0-4c76-b9f3-503d1561d537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:41.213389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-4365ff9e-57a0-4c76-b9f3-503d1561d537 None None] MFA Rules not processed for user `8ccd455b417549e3b00439cee4577d11`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:00:41.214599 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-4365ff9e-57a0-4c76-b9f3-503d1561d537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:41.268120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1259/2558] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:00:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:00:48.349749 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=98 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:00:51.261054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=48 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:00:55.524314 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da1732-a715-447d-8515-b49eebcb8113 service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:55.585500 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da1732-a715-447d-8515-b49eebcb8113 service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:55.590108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e37f3-71a3-4683-bcd9-b63fa9c52d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:55.590771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e37f3-71a3-4683-bcd9-b63fa9c52d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:55.591057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e37f3-71a3-4683-bcd9-b63fa9c52d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:55.591332 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548e37f3-71a3-4683-bcd9-b63fa9c52d58 admin admin] PATH_INFO: `/v3/users/86d7018443694536a7a70f46e59ff93d` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:55.592484 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e37f3-71a3-4683-bcd9-b63fa9c52d58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86d7018443694536a7a70f46e59ff93d)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:55.594392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548e37f3-71a3-4683-bcd9-b63fa9c52d58 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:55.639649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-548e37f3-71a3-4683-bcd9-b63fa9c52d58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86d7018443694536a7a70f46e59ff93d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:55.646438 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-548e37f3-71a3-4683-bcd9-b63fa9c52d58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86d7018443694536a7a70f46e59ff93d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:55.657991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-548e37f3-71a3-4683-bcd9-b63fa9c52d58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86d7018443694536a7a70f46e59ff93d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:55.658699 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-548e37f3-71a3-4683-bcd9-b63fa9c52d58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86d7018443694536a7a70f46e59ff93d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:55.659325 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-548e37f3-71a3-4683-bcd9-b63fa9c52d58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86d7018443694536a7a70f46e59ff93d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:55.672001 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1300/2559] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:00:55 2022] DELETE /identity/v3/users/86d7018443694536a7a70f46e59ff93d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:00:55.677454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440860c-23ac-4b9b-9fa9-3db79e892f8e None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:55.677955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440860c-23ac-4b9b-9fa9-3db79e892f8e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:55.678116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440860c-23ac-4b9b-9fa9-3db79e892f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:55.698002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-f440860c-23ac-4b9b-9fa9-3db79e892f8e None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:00:55.699230 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f440860c-23ac-4b9b-9fa9-3db79e892f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:55.755648 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1260/2560] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:00:55.765266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e37f3-71a3-4683-bcd9-b63fa9c52d58 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:55.766689 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-548e37f3-71a3-4683-bcd9-b63fa9c52d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:55.822632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548e37f3-71a3-4683-bcd9-b63fa9c52d58 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:55.826258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91360540-05f4-4135-8e6c-a5f4496d569a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:55.826892 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91360540-05f4-4135-8e6c-a5f4496d569a service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:55.827175 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91360540-05f4-4135-8e6c-a5f4496d569a service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:55.827448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91360540-05f4-4135-8e6c-a5f4496d569a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:55.836448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91360540-05f4-4135-8e6c-a5f4496d569a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:55.837804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91360540-05f4-4135-8e6c-a5f4496d569a service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:55.847865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1301/2561] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:00:56.118196 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440860c-23ac-4b9b-9fa9-3db79e892f8e None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:56.119760 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f440860c-23ac-4b9b-9fa9-3db79e892f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:56.134437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440860c-23ac-4b9b-9fa9-3db79e892f8e None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:56.139587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392cca1-342e-401a-8b30-57d4109520d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:56.139819 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392cca1-342e-401a-8b30-57d4109520d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:56.140063 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392cca1-342e-401a-8b30-57d4109520d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:56.140397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392cca1-342e-401a-8b30-57d4109520d3 admin admin] PATH_INFO: `/v3/projects/593071d79dca40eca7fc581f74f570ca` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:56.141628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392cca1-342e-401a-8b30-57d4109520d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=593071d79dca40eca7fc581f74f570ca)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:56.143610 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392cca1-342e-401a-8b30-57d4109520d3 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:56.193552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d392cca1-342e-401a-8b30-57d4109520d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '593071d79dca40eca7fc581f74f570ca', 'request_id': 'req-d392cca1-342e-401a-8b30-57d4109520d3'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:56.200520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d392cca1-342e-401a-8b30-57d4109520d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '593071d79dca40eca7fc581f74f570ca', 'request_id': 'req-d392cca1-342e-401a-8b30-57d4109520d3'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:56.207349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d392cca1-342e-401a-8b30-57d4109520d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '593071d79dca40eca7fc581f74f570ca', 'request_id': 'req-d392cca1-342e-401a-8b30-57d4109520d3'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:56.217564 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d392cca1-342e-401a-8b30-57d4109520d3 admin admin] The token cache is being invalidate because project 593071d79dca40eca7fc581f74f570ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:00:56.217852 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d392cca1-342e-401a-8b30-57d4109520d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:56.218495 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d392cca1-342e-401a-8b30-57d4109520d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:56.220671 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1261/2562] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:00:56 2022] DELETE /identity/v3/projects/593071d79dca40eca7fc581f74f570ca => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:00:56.224721 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91360540-05f4-4135-8e6c-a5f4496d569a service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:56.234261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91360540-05f4-4135-8e6c-a5f4496d569a service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:56.252165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:00:56.292105 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11805aa8-b954-45ba-8b0a-b9dff4b66014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:56.292736 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11805aa8-b954-45ba-8b0a-b9dff4b66014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:56.293017 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11805aa8-b954-45ba-8b0a-b9dff4b66014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:56.293290 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11805aa8-b954-45ba-8b0a-b9dff4b66014 admin admin] PATH_INFO: `/v3/users/ae0692d6a427482f9fb907e12442b525` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:56.294458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11805aa8-b954-45ba-8b0a-b9dff4b66014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0692d6a427482f9fb907e12442b525)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:56.296685 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11805aa8-b954-45ba-8b0a-b9dff4b66014 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:56.345977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-11805aa8-b954-45ba-8b0a-b9dff4b66014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0692d6a427482f9fb907e12442b525'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:56.353186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-11805aa8-b954-45ba-8b0a-b9dff4b66014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0692d6a427482f9fb907e12442b525'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:56.365089 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-11805aa8-b954-45ba-8b0a-b9dff4b66014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0692d6a427482f9fb907e12442b525'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:56.365750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-11805aa8-b954-45ba-8b0a-b9dff4b66014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0692d6a427482f9fb907e12442b525'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:56.366366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-11805aa8-b954-45ba-8b0a-b9dff4b66014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0692d6a427482f9fb907e12442b525'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:56.379288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1302/2563] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:00:56 2022] DELETE /identity/v3/users/ae0692d6a427482f9fb907e12442b525 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:00:56.647946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392cca1-342e-401a-8b30-57d4109520d3 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:56.649396 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d392cca1-342e-401a-8b30-57d4109520d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:56.664936 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392cca1-342e-401a-8b30-57d4109520d3 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:56.722223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d6b11-8f4a-46da-902d-a86dae9db8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:56.722904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d6b11-8f4a-46da-902d-a86dae9db8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:56.723185 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d6b11-8f4a-46da-902d-a86dae9db8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:56.723459 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d6b11-8f4a-46da-902d-a86dae9db8dc admin admin] PATH_INFO: `/v3/projects/b72cbb4795634380a7c9e50c71763492` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:56.724615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d6b11-8f4a-46da-902d-a86dae9db8dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b72cbb4795634380a7c9e50c71763492)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:56.726534 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d6b11-8f4a-46da-902d-a86dae9db8dc admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:56.755905 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:00:56.773104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9a7d6b11-8f4a-46da-902d-a86dae9db8dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b72cbb4795634380a7c9e50c71763492', 'request_id': 'req-9a7d6b11-8f4a-46da-902d-a86dae9db8dc'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:56.779957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9a7d6b11-8f4a-46da-902d-a86dae9db8dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b72cbb4795634380a7c9e50c71763492', 'request_id': 'req-9a7d6b11-8f4a-46da-902d-a86dae9db8dc'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:56.786161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9a7d6b11-8f4a-46da-902d-a86dae9db8dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b72cbb4795634380a7c9e50c71763492', 'request_id': 'req-9a7d6b11-8f4a-46da-902d-a86dae9db8dc'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:56.797484 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9a7d6b11-8f4a-46da-902d-a86dae9db8dc admin admin] The token cache is being invalidate because project b72cbb4795634380a7c9e50c71763492 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:00:56.797770 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9a7d6b11-8f4a-46da-902d-a86dae9db8dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:56.798394 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-9a7d6b11-8f4a-46da-902d-a86dae9db8dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:56.799909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1262/2564] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:00:56 2022] DELETE /identity/v3/projects/b72cbb4795634380a7c9e50c71763492 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:00:56.804958 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1686c-0906-48c7-859f-bed25dbb5afb None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:56.805460 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1686c-0906-48c7-859f-bed25dbb5afb None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:56.805575 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1686c-0906-48c7-859f-bed25dbb5afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:56.825394 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-12f1686c-0906-48c7-859f-bed25dbb5afb None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:00:56.826609 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-12f1686c-0906-48c7-859f-bed25dbb5afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:56.883505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1303/2565] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:00:56.892020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e81fa-71ee-47af-98ae-fdc75a211106 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:56.892249 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e81fa-71ee-47af-98ae-fdc75a211106 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:56.892531 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e81fa-71ee-47af-98ae-fdc75a211106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:56.911908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-2c5e81fa-71ee-47af-98ae-fdc75a211106 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:00:56.913109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2c5e81fa-71ee-47af-98ae-fdc75a211106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:56.921569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1263/2566] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:00:56.926247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1686c-0906-48c7-859f-bed25dbb5afb None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:56.935874 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1686c-0906-48c7-859f-bed25dbb5afb None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:56.940991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddb4b9-664f-4351-82a9-61d407e142f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:56.941582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddb4b9-664f-4351-82a9-61d407e142f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:56.941858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddb4b9-664f-4351-82a9-61d407e142f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:56.942131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dddb4b9-664f-4351-82a9-61d407e142f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:56.942842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddb4b9-664f-4351-82a9-61d407e142f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 13:00:56.943144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddb4b9-664f-4351-82a9-61d407e142f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:56.944715 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dddb4b9-664f-4351-82a9-61d407e142f0 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:56.955891 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1304/2567] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 13:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 13:00:56.959992 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e81fa-71ee-47af-98ae-fdc75a211106 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:56.969269 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e81fa-71ee-47af-98ae-fdc75a211106 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:56.973883 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92affb-5900-44e9-9a1f-9088d93c69b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:56.974500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92affb-5900-44e9-9a1f-9088d93c69b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:56.974809 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92affb-5900-44e9-9a1f-9088d93c69b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:56.975087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92affb-5900-44e9-9a1f-9088d93c69b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:56.975756 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92affb-5900-44e9-9a1f-9088d93c69b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:56.977634 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92affb-5900-44e9-9a1f-9088d93c69b8 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:57.005944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1264/2568] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:00:56 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:00:57.010334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddb4b9-664f-4351-82a9-61d407e142f0 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:57.020325 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dddb4b9-664f-4351-82a9-61d407e142f0 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:57.046443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:00:57.080938 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8426ec7-091b-4327-8978-c84216640464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:57.081577 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8426ec7-091b-4327-8978-c84216640464 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:57.081862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8426ec7-091b-4327-8978-c84216640464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:57.082135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8426ec7-091b-4327-8978-c84216640464 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:57.082856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8426ec7-091b-4327-8978-c84216640464 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:57.084751 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8426ec7-091b-4327-8978-c84216640464 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:57.104189 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1305/2569] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:00:57 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:00:57.108363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92affb-5900-44e9-9a1f-9088d93c69b8 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:57.119107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92affb-5900-44e9-9a1f-9088d93c69b8 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:57.123675 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a2bbd-ac47-46cf-9416-0ac870e9eb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:57.124279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a2bbd-ac47-46cf-9416-0ac870e9eb4e admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:57.124591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a2bbd-ac47-46cf-9416-0ac870e9eb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:57.124863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a2bbd-ac47-46cf-9416-0ac870e9eb4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:57.126124 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a2bbd-ac47-46cf-9416-0ac870e9eb4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:57.127732 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a2bbd-ac47-46cf-9416-0ac870e9eb4e admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:57.137424 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1265/2570] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:00:57.141083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8426ec7-091b-4327-8978-c84216640464 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:57.150996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8426ec7-091b-4327-8978-c84216640464 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:57.155587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dff3f2-1ebe-4578-a0a2-236c2b42b4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:57.156188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dff3f2-1ebe-4578-a0a2-236c2b42b4fb admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:57.156488 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dff3f2-1ebe-4578-a0a2-236c2b42b4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:57.156762 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dff3f2-1ebe-4578-a0a2-236c2b42b4fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:57.157447 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dff3f2-1ebe-4578-a0a2-236c2b42b4fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:57.159063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dff3f2-1ebe-4578-a0a2-236c2b42b4fb admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:57.168926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1306/2571] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:00:57.173497 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a2bbd-ac47-46cf-9416-0ac870e9eb4e admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:57.185510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a2bbd-ac47-46cf-9416-0ac870e9eb4e admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:57.191089 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4e2df-9359-4007-9526-5150fe0aee75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:57.191786 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4e2df-9359-4007-9526-5150fe0aee75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:57.192069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4e2df-9359-4007-9526-5150fe0aee75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:57.192344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4e2df-9359-4007-9526-5150fe0aee75 admin admin] PATH_INFO: `/v3/projects/d89e11233b514179b5cbe7a8a0f6a6d6/users/86d10a3eb58f4ccf8af0941c81c234da/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:57.207010 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4e2df-9359-4007-9526-5150fe0aee75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d89e11233b514179b5cbe7a8a0f6a6d6, user_id=86d10a3eb58f4ccf8af0941c81c234da, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:57.209633 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4e2df-9359-4007-9526-5150fe0aee75 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:57.222322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1266/2572] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:00:57 2022] PUT /identity/v3/projects/d89e11233b514179b5cbe7a8a0f6a6d6/users/86d10a3eb58f4ccf8af0941c81c234da/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:00:57.233779 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dafd30-9276-4894-9a15-248192602691 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:57.234064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dafd30-9276-4894-9a15-248192602691 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:57.234338 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dafd30-9276-4894-9a15-248192602691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:57.264767 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-31dafd30-9276-4894-9a15-248192602691 None None] MFA Rules not processed for user `86d10a3eb58f4ccf8af0941c81c234da`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:00:57.266010 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-31dafd30-9276-4894-9a15-248192602691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:57.328939 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1307/2573] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:00:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:00:57.333380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4e2df-9359-4007-9526-5150fe0aee75 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:57.393284 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4e2df-9359-4007-9526-5150fe0aee75 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:57.397831 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f0818-98c2-450e-bfc4-f28dd4a9edd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:57.398433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f0818-98c2-450e-bfc4-f28dd4a9edd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:57.398895 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f0818-98c2-450e-bfc4-f28dd4a9edd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:57.399169 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01f0818-98c2-450e-bfc4-f28dd4a9edd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:57.399820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f0818-98c2-450e-bfc4-f28dd4a9edd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:57.401744 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01f0818-98c2-450e-bfc4-f28dd4a9edd9 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:57.429270 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1267/2574] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:00:57 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:00:57.433331 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dafd30-9276-4894-9a15-248192602691 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:57.443854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dafd30-9276-4894-9a15-248192602691 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:57.499527 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aecc1a-8a2b-434c-ad67-3744f4cb9b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:57.500149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aecc1a-8a2b-434c-ad67-3744f4cb9b29 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:57.500454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aecc1a-8a2b-434c-ad67-3744f4cb9b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:57.500729 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aecc1a-8a2b-434c-ad67-3744f4cb9b29 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:57.501423 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aecc1a-8a2b-434c-ad67-3744f4cb9b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:57.503501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aecc1a-8a2b-434c-ad67-3744f4cb9b29 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:57.525439 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1308/2575] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:00:57 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:00:57.529403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f0818-98c2-450e-bfc4-f28dd4a9edd9 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:57.540496 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01f0818-98c2-450e-bfc4-f28dd4a9edd9 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:57.545527 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee49de1-2973-42e2-8988-ba1a872e0b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:57.546209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee49de1-2973-42e2-8988-ba1a872e0b53 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:57.546613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee49de1-2973-42e2-8988-ba1a872e0b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:57.546954 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee49de1-2973-42e2-8988-ba1a872e0b53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:57.548276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee49de1-2973-42e2-8988-ba1a872e0b53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:57.549917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee49de1-2973-42e2-8988-ba1a872e0b53 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:57.562100 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1268/2576] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 13:00:57.566063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aecc1a-8a2b-434c-ad67-3744f4cb9b29 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:57.576841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aecc1a-8a2b-434c-ad67-3744f4cb9b29 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:57.581453 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b653a0-75e0-4a46-b9a5-ee9adfaea827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:57.582114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b653a0-75e0-4a46-b9a5-ee9adfaea827 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:57.582392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b653a0-75e0-4a46-b9a5-ee9adfaea827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:57.582697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b653a0-75e0-4a46-b9a5-ee9adfaea827 admin admin] PATH_INFO: `/v3/projects/924d994489874ea4be0f42b93c56d78a/users/15c0aaf4693b46668fe1fc8df42c0af7/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:57.598381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b653a0-75e0-4a46-b9a5-ee9adfaea827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=924d994489874ea4be0f42b93c56d78a, user_id=15c0aaf4693b46668fe1fc8df42c0af7, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:57.601062 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b653a0-75e0-4a46-b9a5-ee9adfaea827 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:57.614870 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1309/2577] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:00:57 2022] PUT /identity/v3/projects/924d994489874ea4be0f42b93c56d78a/users/15c0aaf4693b46668fe1fc8df42c0af7/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 13:00:57.618813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee49de1-2973-42e2-8988-ba1a872e0b53 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:57.629376 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee49de1-2973-42e2-8988-ba1a872e0b53 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:57.649925 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:00:57.691754 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a80cb-2df8-4e2d-b4bd-85d0319042c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:57.692371 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a80cb-2df8-4e2d-b4bd-85d0319042c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:57.692645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a80cb-2df8-4e2d-b4bd-85d0319042c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:57.692913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a80cb-2df8-4e2d-b4bd-85d0319042c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:57.693601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a80cb-2df8-4e2d-b4bd-85d0319042c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:57.695224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a80cb-2df8-4e2d-b4bd-85d0319042c9 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:57.705248 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1269/2578] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:00:57.709490 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b653a0-75e0-4a46-b9a5-ee9adfaea827 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:57.715013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:00:57.721919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b653a0-75e0-4a46-b9a5-ee9adfaea827 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:57.727334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005ba8b-8fd5-4b35-b101-41fb06a4fbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:57.728001 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005ba8b-8fd5-4b35-b101-41fb06a4fbf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:57.728280 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005ba8b-8fd5-4b35-b101-41fb06a4fbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:57.728549 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005ba8b-8fd5-4b35-b101-41fb06a4fbf2 admin admin] PATH_INFO: `/v3/projects/924d994489874ea4be0f42b93c56d78a/users/15c0aaf4693b46668fe1fc8df42c0af7/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:57.730622 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005ba8b-8fd5-4b35-b101-41fb06a4fbf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=924d994489874ea4be0f42b93c56d78a, user_id=15c0aaf4693b46668fe1fc8df42c0af7, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:57.733285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005ba8b-8fd5-4b35-b101-41fb06a4fbf2 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:57.755556 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1310/2579] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:00:57 2022] PUT /identity/v3/projects/924d994489874ea4be0f42b93c56d78a/users/15c0aaf4693b46668fe1fc8df42c0af7/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:00:57.769277 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c219572-cf81-47a2-a008-ec3669f54a4f None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:57.769771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c219572-cf81-47a2-a008-ec3669f54a4f None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:57.769891 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c219572-cf81-47a2-a008-ec3669f54a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:57.806720 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-2c219572-cf81-47a2-a008-ec3669f54a4f None None] MFA Rules not processed for user `15c0aaf4693b46668fe1fc8df42c0af7`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:00:57.807921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2c219572-cf81-47a2-a008-ec3669f54a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:57.896674 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1270/2580] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:00:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:00:57.907835 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005ba8b-8fd5-4b35-b101-41fb06a4fbf2 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:57.909647 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-4005ba8b-8fd5-4b35-b101-41fb06a4fbf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:00:57.968924 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005ba8b-8fd5-4b35-b101-41fb06a4fbf2 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:57.973224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910191c4-ca97-46f6-96b8-bc91058b4bd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:57.973833 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910191c4-ca97-46f6-96b8-bc91058b4bd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:57.974114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910191c4-ca97-46f6-96b8-bc91058b4bd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:57.974384 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910191c4-ca97-46f6-96b8-bc91058b4bd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:57.983627 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910191c4-ca97-46f6-96b8-bc91058b4bd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:57.985092 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910191c4-ca97-46f6-96b8-bc91058b4bd8 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:57.994560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1311/2581] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:00:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:00:59.738675 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c219572-cf81-47a2-a008-ec3669f54a4f None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:00:59.823693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:00:59.827854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c219572-cf81-47a2-a008-ec3669f54a4f None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:00:59.833512 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154f663-65df-44a7-86a0-85221c848818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:00:59.834161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154f663-65df-44a7-86a0-85221c848818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:59.834436 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154f663-65df-44a7-86a0-85221c848818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:59.834740 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154f663-65df-44a7-86a0-85221c848818 admin admin] PATH_INFO: `/v3/users/86d10a3eb58f4ccf8af0941c81c234da` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:59.835929 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154f663-65df-44a7-86a0-85221c848818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86d10a3eb58f4ccf8af0941c81c234da)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:00:59.837811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154f663-65df-44a7-86a0-85221c848818 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:00:59.895257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1154f663-65df-44a7-86a0-85221c848818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86d10a3eb58f4ccf8af0941c81c234da'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:59.907789 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1154f663-65df-44a7-86a0-85221c848818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86d10a3eb58f4ccf8af0941c81c234da'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:59.917049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1154f663-65df-44a7-86a0-85221c848818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86d10a3eb58f4ccf8af0941c81c234da'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:59.917784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1154f663-65df-44a7-86a0-85221c848818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86d10a3eb58f4ccf8af0941c81c234da'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:59.925924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1154f663-65df-44a7-86a0-85221c848818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86d10a3eb58f4ccf8af0941c81c234da'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:00:59.932150 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1271/2582] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:00:59 2022] DELETE /identity/v3/users/86d10a3eb58f4ccf8af0941c81c234da => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:00:59.936768 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5a365-2418-44bb-b57b-77438cf50854 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:00:59.937329 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5a365-2418-44bb-b57b-77438cf50854 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:00:59.937429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5a365-2418-44bb-b57b-77438cf50854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:00:59.966139 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-52c5a365-2418-44bb-b57b-77438cf50854 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:00:59.967634 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-52c5a365-2418-44bb-b57b-77438cf50854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:00.030000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1312/2583] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:00:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:00.039796 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154f663-65df-44a7-86a0-85221c848818 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:00.041328 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1154f663-65df-44a7-86a0-85221c848818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:00.137190 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154f663-65df-44a7-86a0-85221c848818 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:00.141087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaff434-af92-4f12-91f3-9bed0871156b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:00.141709 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaff434-af92-4f12-91f3-9bed0871156b service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:00.141997 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaff434-af92-4f12-91f3-9bed0871156b service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:00.142273 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaff434-af92-4f12-91f3-9bed0871156b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:00.152229 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaff434-af92-4f12-91f3-9bed0871156b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:00.153900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaff434-af92-4f12-91f3-9bed0871156b service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:00.164529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1272/2584] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:01:00.269792 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5a365-2418-44bb-b57b-77438cf50854 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:00.271514 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-52c5a365-2418-44bb-b57b-77438cf50854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:00.284066 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5a365-2418-44bb-b57b-77438cf50854 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:00.287725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72b7dd-66f8-46bc-bae4-e2fc920a5cf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:00.288486 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72b7dd-66f8-46bc-bae4-e2fc920a5cf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:00.289013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72b7dd-66f8-46bc-bae4-e2fc920a5cf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:00.289379 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72b7dd-66f8-46bc-bae4-e2fc920a5cf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:00.295777 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ad72b7dd-66f8-46bc-bae4-e2fc920a5cf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:00.365490 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72b7dd-66f8-46bc-bae4-e2fc920a5cf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:00.366995 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad72b7dd-66f8-46bc-bae4-e2fc920a5cf7 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:00.376867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1313/2585] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:01:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:01:00.528102 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaff434-af92-4f12-91f3-9bed0871156b service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:00.529712 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2eaff434-af92-4f12-91f3-9bed0871156b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:00.549875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaff434-af92-4f12-91f3-9bed0871156b service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:00.556183 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:00.556833 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:00.557146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:00.557373 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681 admin admin] PATH_INFO: `/v3/projects/d89e11233b514179b5cbe7a8a0f6a6d6` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:00.558587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d89e11233b514179b5cbe7a8a0f6a6d6)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:00.560591 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:00.629626 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd89e11233b514179b5cbe7a8a0f6a6d6', 'request_id': 'req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:00.639818 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd89e11233b514179b5cbe7a8a0f6a6d6', 'request_id': 'req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:00.641690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:00.651850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd89e11233b514179b5cbe7a8a0f6a6d6', 'request_id': 'req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:00.665115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681 admin admin] The token cache is being invalidate because project d89e11233b514179b5cbe7a8a0f6a6d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:01:00.665509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:00.666536 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:00.668062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1273/2586] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:01:00 2022] DELETE /identity/v3/projects/d89e11233b514179b5cbe7a8a0f6a6d6 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:00.672834 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72b7dd-66f8-46bc-bae4-e2fc920a5cf7 service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:00.682082 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:00.690073 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad72b7dd-66f8-46bc-bae4-e2fc920a5cf7 service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:00.752169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bb16c-b61a-4bcb-b0e5-6a805d07aa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:00.753360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bb16c-b61a-4bcb-b0e5-6a805d07aa77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:00.753886 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bb16c-b61a-4bcb-b0e5-6a805d07aa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:00.754386 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bb16c-b61a-4bcb-b0e5-6a805d07aa77 admin admin] PATH_INFO: `/v3/users/15c0aaf4693b46668fe1fc8df42c0af7` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:00.755976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0bb16c-b61a-4bcb-b0e5-6a805d07aa77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15c0aaf4693b46668fe1fc8df42c0af7)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:00.758093 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0bb16c-b61a-4bcb-b0e5-6a805d07aa77 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:00.811596 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fa0bb16c-b61a-4bcb-b0e5-6a805d07aa77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15c0aaf4693b46668fe1fc8df42c0af7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:00.819046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fa0bb16c-b61a-4bcb-b0e5-6a805d07aa77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15c0aaf4693b46668fe1fc8df42c0af7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:00.835583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fa0bb16c-b61a-4bcb-b0e5-6a805d07aa77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15c0aaf4693b46668fe1fc8df42c0af7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:00.836547 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fa0bb16c-b61a-4bcb-b0e5-6a805d07aa77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15c0aaf4693b46668fe1fc8df42c0af7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:00.837312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fa0bb16c-b61a-4bcb-b0e5-6a805d07aa77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15c0aaf4693b46668fe1fc8df42c0af7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:00.853546 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1314/2587] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:01:00 2022] DELETE /identity/v3/users/15c0aaf4693b46668fe1fc8df42c0af7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:01.200220 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:01.201819 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:01.223312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dde0d-c1ac-4dcf-b7e8-f9a47746c681 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:01.298804 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ca28a-dc86-4a92-8285-7f17502197c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:01.299494 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ca28a-dc86-4a92-8285-7f17502197c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:01.299773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ca28a-dc86-4a92-8285-7f17502197c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:01.300086 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ca28a-dc86-4a92-8285-7f17502197c3 admin admin] PATH_INFO: `/v3/projects/924d994489874ea4be0f42b93c56d78a` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:01.301317 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ca28a-dc86-4a92-8285-7f17502197c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=924d994489874ea4be0f42b93c56d78a)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:01.303294 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2ca28a-dc86-4a92-8285-7f17502197c3 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:01.357420 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cb2ca28a-dc86-4a92-8285-7f17502197c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '924d994489874ea4be0f42b93c56d78a', 'request_id': 'req-cb2ca28a-dc86-4a92-8285-7f17502197c3'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:01.365788 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cb2ca28a-dc86-4a92-8285-7f17502197c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '924d994489874ea4be0f42b93c56d78a', 'request_id': 'req-cb2ca28a-dc86-4a92-8285-7f17502197c3'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:01.372467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cb2ca28a-dc86-4a92-8285-7f17502197c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '924d994489874ea4be0f42b93c56d78a', 'request_id': 'req-cb2ca28a-dc86-4a92-8285-7f17502197c3'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:01.387277 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cb2ca28a-dc86-4a92-8285-7f17502197c3 admin admin] The token cache is being invalidate because project 924d994489874ea4be0f42b93c56d78a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:01:01.387821 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cb2ca28a-dc86-4a92-8285-7f17502197c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:01.388448 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-cb2ca28a-dc86-4a92-8285-7f17502197c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:01.390422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1274/2588] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:01:01 2022] DELETE /identity/v3/projects/924d994489874ea4be0f42b93c56d78a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:01.395531 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74558cb2-a677-4210-8f2c-c1257404be6b None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:01.396051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74558cb2-a677-4210-8f2c-c1257404be6b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:01.396326 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74558cb2-a677-4210-8f2c-c1257404be6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:01.460784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-74558cb2-a677-4210-8f2c-c1257404be6b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:01.462183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-74558cb2-a677-4210-8f2c-c1257404be6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:01.526083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:01.541881 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1315/2589] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 13:01:01.551057 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bc6e0-e8a5-4eae-9208-5208a6f5bcb6 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:01.551576 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bc6e0-e8a5-4eae-9208-5208a6f5bcb6 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:01.551854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bc6e0-e8a5-4eae-9208-5208a6f5bcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:01.574720 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-e95bc6e0-e8a5-4eae-9208-5208a6f5bcb6 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:01.575973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e95bc6e0-e8a5-4eae-9208-5208a6f5bcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:01.585205 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1275/2590] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:01.589266 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74558cb2-a677-4210-8f2c-c1257404be6b None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:01.599960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74558cb2-a677-4210-8f2c-c1257404be6b None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:01.605850 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc28fca-afec-4f21-989e-e0df635ed289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:01.609345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc28fca-afec-4f21-989e-e0df635ed289 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:01.609345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc28fca-afec-4f21-989e-e0df635ed289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:01.609345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc28fca-afec-4f21-989e-e0df635ed289 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:01.609345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc28fca-afec-4f21-989e-e0df635ed289 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 13:01:01.609345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc28fca-afec-4f21-989e-e0df635ed289 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:01.611883 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc28fca-afec-4f21-989e-e0df635ed289 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:01.627078 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1316/2591] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 13:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 13:01:01.631325 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95bc6e0-e8a5-4eae-9208-5208a6f5bcb6 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:01.648511 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95bc6e0-e8a5-4eae-9208-5208a6f5bcb6 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:01.653546 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fb438-2a24-4f0f-a857-566834544ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:01.654172 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fb438-2a24-4f0f-a857-566834544ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:01.654449 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fb438-2a24-4f0f-a857-566834544ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:01.654773 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fb438-2a24-4f0f-a857-566834544ac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:01.655635 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fb438-2a24-4f0f-a857-566834544ac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:01.657570 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1fb438-2a24-4f0f-a857-566834544ac1 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:01.694529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1276/2592] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:01:01 2022] POST /identity/v3/projects => generated 401 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:01:01.696974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc28fca-afec-4f21-989e-e0df635ed289 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:01.709878 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc28fca-afec-4f21-989e-e0df635ed289 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:01.786498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e030d9e-621c-4d69-bd18-51deaebeb15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:01.787264 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e030d9e-621c-4d69-bd18-51deaebeb15e admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:01.787654 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e030d9e-621c-4d69-bd18-51deaebeb15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:01.788054 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e030d9e-621c-4d69-bd18-51deaebeb15e admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:01.788782 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e030d9e-621c-4d69-bd18-51deaebeb15e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:01.790759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e030d9e-621c-4d69-bd18-51deaebeb15e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:01.818368 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1317/2593] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:01:01 2022] POST /identity/v3/users => generated 301 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:01:01.822708 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fb438-2a24-4f0f-a857-566834544ac1 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:01.834917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fb438-2a24-4f0f-a857-566834544ac1 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:01.840972 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e450deb-e9d8-4525-96a7-3c135bd36747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:01.841493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e450deb-e9d8-4525-96a7-3c135bd36747 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:01.841785 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e450deb-e9d8-4525-96a7-3c135bd36747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:01.842134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e450deb-e9d8-4525-96a7-3c135bd36747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:01.842808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e450deb-e9d8-4525-96a7-3c135bd36747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:01.844668 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e450deb-e9d8-4525-96a7-3c135bd36747 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:01.862589 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1277/2594] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:01:01.866609 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e030d9e-621c-4d69-bd18-51deaebeb15e admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:01.878075 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e030d9e-621c-4d69-bd18-51deaebeb15e admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:01.884748 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acc2bc-09f4-48cb-9d4a-ffe2c0430199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:01.886590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acc2bc-09f4-48cb-9d4a-ffe2c0430199 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:01.886590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acc2bc-09f4-48cb-9d4a-ffe2c0430199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:01.886854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acc2bc-09f4-48cb-9d4a-ffe2c0430199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:01.887663 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7acc2bc-09f4-48cb-9d4a-ffe2c0430199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:01.889306 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7acc2bc-09f4-48cb-9d4a-ffe2c0430199 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:01.905663 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1318/2595] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:01:01.909946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e450deb-e9d8-4525-96a7-3c135bd36747 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:01.920960 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e450deb-e9d8-4525-96a7-3c135bd36747 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:01.927083 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12241ffc-a865-4cdc-9200-998cd3fb20d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:01.927771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12241ffc-a865-4cdc-9200-998cd3fb20d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:01.928052 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12241ffc-a865-4cdc-9200-998cd3fb20d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:01.928330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12241ffc-a865-4cdc-9200-998cd3fb20d9 admin admin] PATH_INFO: `/v3/projects/0fcbcf201d68415f825270eaf203dfc0/users/76449ae20a504578af1b20b977d157d4/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:01.946263 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12241ffc-a865-4cdc-9200-998cd3fb20d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fcbcf201d68415f825270eaf203dfc0, user_id=76449ae20a504578af1b20b977d157d4, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:01.948965 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12241ffc-a865-4cdc-9200-998cd3fb20d9 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:01.966328 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1278/2596] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:01:01 2022] PUT /identity/v3/projects/0fcbcf201d68415f825270eaf203dfc0/users/76449ae20a504578af1b20b977d157d4/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:01.976522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc50eb-6db8-43de-b291-b6f417e6a6fb None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:01.976833 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc50eb-6db8-43de-b291-b6f417e6a6fb None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:01.977175 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc50eb-6db8-43de-b291-b6f417e6a6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:02.005383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-05bc50eb-6db8-43de-b291-b6f417e6a6fb None None] MFA Rules not processed for user `76449ae20a504578af1b20b977d157d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:02.006667 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-05bc50eb-6db8-43de-b291-b6f417e6a6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:02.096697 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1319/2597] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:01 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:02.101518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12241ffc-a865-4cdc-9200-998cd3fb20d9 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:02.121440 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:02.210101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12241ffc-a865-4cdc-9200-998cd3fb20d9 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:02.215947 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec422b-5b76-4f64-a69d-acd57fed3064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:02.217143 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec422b-5b76-4f64-a69d-acd57fed3064 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:02.217819 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec422b-5b76-4f64-a69d-acd57fed3064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:02.218148 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec422b-5b76-4f64-a69d-acd57fed3064 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:02.220037 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec422b-5b76-4f64-a69d-acd57fed3064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:02.222641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec422b-5b76-4f64-a69d-acd57fed3064 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:02.270291 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc50eb-6db8-43de-b291-b6f417e6a6fb None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:02.274684 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1279/2598] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:01:02 2022] POST /identity/v3/projects => generated 401 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:01:02.286733 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc50eb-6db8-43de-b291-b6f417e6a6fb None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:02.350658 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:02.362343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f21fa5-fbde-421c-8e96-1a932a18aafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:02.363339 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f21fa5-fbde-421c-8e96-1a932a18aafe admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:02.363515 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f21fa5-fbde-421c-8e96-1a932a18aafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:02.363899 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f21fa5-fbde-421c-8e96-1a932a18aafe admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:02.364913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f21fa5-fbde-421c-8e96-1a932a18aafe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:02.368356 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f21fa5-fbde-421c-8e96-1a932a18aafe admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:02.406540 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1320/2599] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:01:02 2022] POST /identity/v3/users => generated 307 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:01:02.410279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec422b-5b76-4f64-a69d-acd57fed3064 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:02.433498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec422b-5b76-4f64-a69d-acd57fed3064 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:02.446876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891fa00-4e97-4c54-9ecc-1dc8418aa1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:02.448437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891fa00-4e97-4c54-9ecc-1dc8418aa1d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:02.449091 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891fa00-4e97-4c54-9ecc-1dc8418aa1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:02.449762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891fa00-4e97-4c54-9ecc-1dc8418aa1d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:02.450853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891fa00-4e97-4c54-9ecc-1dc8418aa1d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:02.453838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b891fa00-4e97-4c54-9ecc-1dc8418aa1d3 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:02.469777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1280/2600] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:01:02.471274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f21fa5-fbde-421c-8e96-1a932a18aafe admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:02.485983 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f21fa5-fbde-421c-8e96-1a932a18aafe admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:02.493368 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad3daa-9856-418f-81d6-86c94c6055da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:02.495588 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad3daa-9856-418f-81d6-86c94c6055da admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:02.495588 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad3daa-9856-418f-81d6-86c94c6055da admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:02.495947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad3daa-9856-418f-81d6-86c94c6055da admin admin] PATH_INFO: `/v3/projects/ec7505cf63144c39b835ef58dedca1be/users/aa3d707d57894ea4bf8bd1e4f408b2ed/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:02.517216 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad3daa-9856-418f-81d6-86c94c6055da admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec7505cf63144c39b835ef58dedca1be, user_id=aa3d707d57894ea4bf8bd1e4f408b2ed, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:02.520018 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad3daa-9856-418f-81d6-86c94c6055da admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:02.541167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1321/2601] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:01:02 2022] PUT /identity/v3/projects/ec7505cf63144c39b835ef58dedca1be/users/aa3d707d57894ea4bf8bd1e4f408b2ed/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:02.542461 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891fa00-4e97-4c54-9ecc-1dc8418aa1d3 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:02.559687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891fa00-4e97-4c54-9ecc-1dc8418aa1d3 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:02.643518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e326be-b7cc-4c64-a10c-b1349e8d6c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:02.644154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e326be-b7cc-4c64-a10c-b1349e8d6c76 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:02.644436 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e326be-b7cc-4c64-a10c-b1349e8d6c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:02.644709 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e326be-b7cc-4c64-a10c-b1349e8d6c76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:02.645468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e326be-b7cc-4c64-a10c-b1349e8d6c76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:02.647483 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e326be-b7cc-4c64-a10c-b1349e8d6c76 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:02.672946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1281/2602] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:01:02.673610 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad3daa-9856-418f-81d6-86c94c6055da admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:02.689232 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad3daa-9856-418f-81d6-86c94c6055da admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:02.695421 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8abe25-434f-4c52-85b6-3a9c951426ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:02.696362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8abe25-434f-4c52-85b6-3a9c951426ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:02.696647 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8abe25-434f-4c52-85b6-3a9c951426ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:02.696928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8abe25-434f-4c52-85b6-3a9c951426ef admin admin] PATH_INFO: `/v3/projects/ec7505cf63144c39b835ef58dedca1be/users/aa3d707d57894ea4bf8bd1e4f408b2ed/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:02.701044 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8abe25-434f-4c52-85b6-3a9c951426ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec7505cf63144c39b835ef58dedca1be, user_id=aa3d707d57894ea4bf8bd1e4f408b2ed, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:02.706284 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8abe25-434f-4c52-85b6-3a9c951426ef admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:02.723008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1322/2603] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:01:02 2022] PUT /identity/v3/projects/ec7505cf63144c39b835ef58dedca1be/users/aa3d707d57894ea4bf8bd1e4f408b2ed/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 13:01:02.733099 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef190d1b-51ea-4ecb-984e-b7d2057bfe37 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:02.733289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef190d1b-51ea-4ecb-984e-b7d2057bfe37 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:02.733544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef190d1b-51ea-4ecb-984e-b7d2057bfe37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:02.775666 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-ef190d1b-51ea-4ecb-984e-b7d2057bfe37 None None] MFA Rules not processed for user `aa3d707d57894ea4bf8bd1e4f408b2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:02.776933 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ef190d1b-51ea-4ecb-984e-b7d2057bfe37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:02.892467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1282/2604] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:02 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:02.897342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127a86e-96aa-4933-a754-8f6fad9e0f1e None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:02.897633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127a86e-96aa-4933-a754-8f6fad9e0f1e None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:02.897909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0127a86e-96aa-4933-a754-8f6fad9e0f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:02.900017 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef190d1b-51ea-4ecb-984e-b7d2057bfe37 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:02.901644 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ef190d1b-51ea-4ecb-984e-b7d2057bfe37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:02.913382 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:02.924642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-0127a86e-96aa-4933-a754-8f6fad9e0f1e None None] MFA Rules not processed for user `fd489ef668494ec98cf9d167a1ded3ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:02.926436 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0127a86e-96aa-4933-a754-8f6fad9e0f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:02.997491 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef190d1b-51ea-4ecb-984e-b7d2057bfe37 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:03.003682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d293115-8d70-46cc-8e60-522db7c46e9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:03.004264 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d293115-8d70-46cc-8e60-522db7c46e9d service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:03.004547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d293115-8d70-46cc-8e60-522db7c46e9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:03.004815 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d293115-8d70-46cc-8e60-522db7c46e9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:03.015801 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1323/2605] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 13:01:03.017183 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d293115-8d70-46cc-8e60-522db7c46e9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:03.018874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d293115-8d70-46cc-8e60-522db7c46e9d service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:03.029998 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1283/2606] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:01:02 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:01:03.116625 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127a86e-96aa-4933-a754-8f6fad9e0f1e None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:03.127533 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0127a86e-96aa-4933-a754-8f6fad9e0f1e None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:03.132035 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ef074-98f7-4e3a-8740-44b3621268b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:03.132679 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ef074-98f7-4e3a-8740-44b3621268b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:03.132957 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ef074-98f7-4e3a-8740-44b3621268b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:03.133240 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ef074-98f7-4e3a-8740-44b3621268b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:03.201639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ef074-98f7-4e3a-8740-44b3621268b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:03.203254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ef074-98f7-4e3a-8740-44b3621268b7 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:03.214252 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1324/2607] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:01:03 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:01:07.441988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d293115-8d70-46cc-8e60-522db7c46e9d service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:07.506529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d293115-8d70-46cc-8e60-522db7c46e9d service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:07.511050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed99923-8679-4381-930b-06a8dba57f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:07.511682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed99923-8679-4381-930b-06a8dba57f49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:07.511973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed99923-8679-4381-930b-06a8dba57f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:07.512251 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed99923-8679-4381-930b-06a8dba57f49 admin admin] PATH_INFO: `/v3/users/76449ae20a504578af1b20b977d157d4` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:07.513413 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed99923-8679-4381-930b-06a8dba57f49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76449ae20a504578af1b20b977d157d4)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:07.515379 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed99923-8679-4381-930b-06a8dba57f49 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:07.687620 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-6ed99923-8679-4381-930b-06a8dba57f49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76449ae20a504578af1b20b977d157d4'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:07.700168 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-6ed99923-8679-4381-930b-06a8dba57f49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76449ae20a504578af1b20b977d157d4'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:07.706910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-6ed99923-8679-4381-930b-06a8dba57f49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76449ae20a504578af1b20b977d157d4'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:07.707704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-6ed99923-8679-4381-930b-06a8dba57f49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76449ae20a504578af1b20b977d157d4'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:07.714808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-6ed99923-8679-4381-930b-06a8dba57f49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76449ae20a504578af1b20b977d157d4'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:07.720595 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1284/2608] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:01:07 2022] DELETE /identity/v3/users/76449ae20a504578af1b20b977d157d4 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:07.725508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd052c14-2aed-4fc5-bb28-7c16d986428f None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:07.726010 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd052c14-2aed-4fc5-bb28-7c16d986428f None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:07.726131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd052c14-2aed-4fc5-bb28-7c16d986428f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:07.746055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-fd052c14-2aed-4fc5-bb28-7c16d986428f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:07.747391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-fd052c14-2aed-4fc5-bb28-7c16d986428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:07.799172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:07.819924 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1325/2609] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:07.831473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed99923-8679-4381-930b-06a8dba57f49 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:07.832868 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-6ed99923-8679-4381-930b-06a8dba57f49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:07.869197 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:07.901144 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed99923-8679-4381-930b-06a8dba57f49 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:07.905153 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a6bc1-3c6a-4f5a-94b5-ad8b72f257a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:07.905772 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a6bc1-3c6a-4f5a-94b5-ad8b72f257a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:07.906047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a6bc1-3c6a-4f5a-94b5-ad8b72f257a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:07.906314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a6bc1-3c6a-4f5a-94b5-ad8b72f257a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:07.916453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a6bc1-3c6a-4f5a-94b5-ad8b72f257a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:07.917858 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a6bc1-3c6a-4f5a-94b5-ad8b72f257a8 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:07.928552 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1285/2610] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:01:08.372681 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd052c14-2aed-4fc5-bb28-7c16d986428f None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:08.374283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-fd052c14-2aed-4fc5-bb28-7c16d986428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:08.412192 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd052c14-2aed-4fc5-bb28-7c16d986428f None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:08.420215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580566e8-0e39-490e-ba89-fcb2c2f17e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:08.420878 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580566e8-0e39-490e-ba89-fcb2c2f17e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:08.421157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580566e8-0e39-490e-ba89-fcb2c2f17e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:08.421591 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580566e8-0e39-490e-ba89-fcb2c2f17e0a admin admin] PATH_INFO: `/v3/projects/0fcbcf201d68415f825270eaf203dfc0` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:08.422903 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580566e8-0e39-490e-ba89-fcb2c2f17e0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fcbcf201d68415f825270eaf203dfc0)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:08.426578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580566e8-0e39-490e-ba89-fcb2c2f17e0a admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:08.491131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-580566e8-0e39-490e-ba89-fcb2c2f17e0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fcbcf201d68415f825270eaf203dfc0', 'request_id': 'req-580566e8-0e39-490e-ba89-fcb2c2f17e0a'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:08.499804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-580566e8-0e39-490e-ba89-fcb2c2f17e0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fcbcf201d68415f825270eaf203dfc0', 'request_id': 'req-580566e8-0e39-490e-ba89-fcb2c2f17e0a'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:08.509875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-580566e8-0e39-490e-ba89-fcb2c2f17e0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fcbcf201d68415f825270eaf203dfc0', 'request_id': 'req-580566e8-0e39-490e-ba89-fcb2c2f17e0a'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:08.529005 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-580566e8-0e39-490e-ba89-fcb2c2f17e0a admin admin] The token cache is being invalidate because project 0fcbcf201d68415f825270eaf203dfc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:01:08.529711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-580566e8-0e39-490e-ba89-fcb2c2f17e0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:08.530698 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-580566e8-0e39-490e-ba89-fcb2c2f17e0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:08.533107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1326/2611] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:01:08 2022] DELETE /identity/v3/projects/0fcbcf201d68415f825270eaf203dfc0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:08.537632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a6bc1-3c6a-4f5a-94b5-ad8b72f257a8 service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:08.553058 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a6bc1-3c6a-4f5a-94b5-ad8b72f257a8 service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:08.636769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a8883-69b9-4d6a-8bdc-d50ecae6262d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:08.637242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a8883-69b9-4d6a-8bdc-d50ecae6262d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:08.637514 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a8883-69b9-4d6a-8bdc-d50ecae6262d admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:08.637784 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41a8883-69b9-4d6a-8bdc-d50ecae6262d admin admin] PATH_INFO: `/v3/users/aa3d707d57894ea4bf8bd1e4f408b2ed` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:08.638980 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a8883-69b9-4d6a-8bdc-d50ecae6262d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa3d707d57894ea4bf8bd1e4f408b2ed)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:08.640859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41a8883-69b9-4d6a-8bdc-d50ecae6262d admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:08.674416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580566e8-0e39-490e-ba89-fcb2c2f17e0a admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:08.675897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-580566e8-0e39-490e-ba89-fcb2c2f17e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:08.692285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580566e8-0e39-490e-ba89-fcb2c2f17e0a admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:08.704596 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e1444-547b-4c44-b192-6f633324b6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:08.705253 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e1444-547b-4c44-b192-6f633324b6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:08.705527 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e1444-547b-4c44-b192-6f633324b6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:08.705793 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e1444-547b-4c44-b192-6f633324b6b1 admin admin] PATH_INFO: `/v3/users/6d333c7a05af409f811c4382f048d1fb` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:08.708050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e1444-547b-4c44-b192-6f633324b6b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d333c7a05af409f811c4382f048d1fb)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:08.710062 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e1444-547b-4c44-b192-6f633324b6b1 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:08.716130 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d41a8883-69b9-4d6a-8bdc-d50ecae6262d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa3d707d57894ea4bf8bd1e4f408b2ed'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:08.729106 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d41a8883-69b9-4d6a-8bdc-d50ecae6262d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa3d707d57894ea4bf8bd1e4f408b2ed'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:08.737019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d41a8883-69b9-4d6a-8bdc-d50ecae6262d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa3d707d57894ea4bf8bd1e4f408b2ed'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:08.737637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d41a8883-69b9-4d6a-8bdc-d50ecae6262d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa3d707d57894ea4bf8bd1e4f408b2ed'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:08.744915 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d41a8883-69b9-4d6a-8bdc-d50ecae6262d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa3d707d57894ea4bf8bd1e4f408b2ed'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:08.751710 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1286/2612] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:01:08 2022] DELETE /identity/v3/users/aa3d707d57894ea4bf8bd1e4f408b2ed => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:08.767772 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-933e1444-547b-4c44-b192-6f633324b6b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d333c7a05af409f811c4382f048d1fb'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:08.774839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-933e1444-547b-4c44-b192-6f633324b6b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d333c7a05af409f811c4382f048d1fb'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:08.788713 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-933e1444-547b-4c44-b192-6f633324b6b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d333c7a05af409f811c4382f048d1fb'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:08.789570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-933e1444-547b-4c44-b192-6f633324b6b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d333c7a05af409f811c4382f048d1fb'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:08.790251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-933e1444-547b-4c44-b192-6f633324b6b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d333c7a05af409f811c4382f048d1fb'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:08.806641 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1327/2613] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:01:08 2022] DELETE /identity/v3/users/6d333c7a05af409f811c4382f048d1fb => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 13:01:09.075910 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a8883-69b9-4d6a-8bdc-d50ecae6262d admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:09.077463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d41a8883-69b9-4d6a-8bdc-d50ecae6262d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:09.135905 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e1444-547b-4c44-b192-6f633324b6b1 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:09.137574 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-933e1444-547b-4c44-b192-6f633324b6b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:09.153488 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:09.173223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41a8883-69b9-4d6a-8bdc-d50ecae6262d admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:09.193045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:09.210306 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e1444-547b-4c44-b192-6f633324b6b1 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:09.214996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e539bc2-504c-41ff-aa7b-a1ac4e265d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:09.215737 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e539bc2-504c-41ff-aa7b-a1ac4e265d64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.216071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e539bc2-504c-41ff-aa7b-a1ac4e265d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.216407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e539bc2-504c-41ff-aa7b-a1ac4e265d64 admin admin] PATH_INFO: `/v3/projects/ec7505cf63144c39b835ef58dedca1be` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.217618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e539bc2-504c-41ff-aa7b-a1ac4e265d64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec7505cf63144c39b835ef58dedca1be)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:09.219656 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e539bc2-504c-41ff-aa7b-a1ac4e265d64 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:09.231047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618abb9e-080e-4d03-abcb-252ed76c59ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:09.231698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618abb9e-080e-4d03-abcb-252ed76c59ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.231983 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618abb9e-080e-4d03-abcb-252ed76c59ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.232261 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618abb9e-080e-4d03-abcb-252ed76c59ae admin admin] PATH_INFO: `/v3/projects/5f4f8e3907cd44e687d18f5a890ef278` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.233492 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618abb9e-080e-4d03-abcb-252ed76c59ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f4f8e3907cd44e687d18f5a890ef278)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:09.235947 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618abb9e-080e-4d03-abcb-252ed76c59ae admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:09.265765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2e539bc2-504c-41ff-aa7b-a1ac4e265d64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec7505cf63144c39b835ef58dedca1be', 'request_id': 'req-2e539bc2-504c-41ff-aa7b-a1ac4e265d64'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:09.273390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2e539bc2-504c-41ff-aa7b-a1ac4e265d64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec7505cf63144c39b835ef58dedca1be', 'request_id': 'req-2e539bc2-504c-41ff-aa7b-a1ac4e265d64'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:09.279840 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2e539bc2-504c-41ff-aa7b-a1ac4e265d64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec7505cf63144c39b835ef58dedca1be', 'request_id': 'req-2e539bc2-504c-41ff-aa7b-a1ac4e265d64'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:09.285572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-618abb9e-080e-4d03-abcb-252ed76c59ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f4f8e3907cd44e687d18f5a890ef278', 'request_id': 'req-618abb9e-080e-4d03-abcb-252ed76c59ae'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:09.291473 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2e539bc2-504c-41ff-aa7b-a1ac4e265d64 admin admin] The token cache is being invalidate because project ec7505cf63144c39b835ef58dedca1be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:01:09.291758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2e539bc2-504c-41ff-aa7b-a1ac4e265d64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:09.292439 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2e539bc2-504c-41ff-aa7b-a1ac4e265d64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:09.292949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-618abb9e-080e-4d03-abcb-252ed76c59ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f4f8e3907cd44e687d18f5a890ef278', 'request_id': 'req-618abb9e-080e-4d03-abcb-252ed76c59ae'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:09.293908 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1328/2614] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:01:09 2022] DELETE /identity/v3/projects/ec7505cf63144c39b835ef58dedca1be => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:09.299600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4289049-79e0-4965-ba06-fc8e65e1fd9a None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.299887 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4289049-79e0-4965-ba06-fc8e65e1fd9a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.300176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4289049-79e0-4965-ba06-fc8e65e1fd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.303934 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-618abb9e-080e-4d03-abcb-252ed76c59ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f4f8e3907cd44e687d18f5a890ef278', 'request_id': 'req-618abb9e-080e-4d03-abcb-252ed76c59ae'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:09.315785 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-618abb9e-080e-4d03-abcb-252ed76c59ae admin admin] The token cache is being invalidate because project 5f4f8e3907cd44e687d18f5a890ef278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:01:09.316075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-618abb9e-080e-4d03-abcb-252ed76c59ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:09.316739 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-618abb9e-080e-4d03-abcb-252ed76c59ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:09.318218 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1287/2615] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:01:09 2022] DELETE /identity/v3/projects/5f4f8e3907cd44e687d18f5a890ef278 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:09.323780 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-b4289049-79e0-4965-ba06-fc8e65e1fd9a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:09.324166 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87145ace-6a97-4377-ae9a-3f619f71ae95 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.324453 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87145ace-6a97-4377-ae9a-3f619f71ae95 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.324729 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87145ace-6a97-4377-ae9a-3f619f71ae95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.325000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b4289049-79e0-4965-ba06-fc8e65e1fd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:09.343410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-87145ace-6a97-4377-ae9a-3f619f71ae95 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:09.345076 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-87145ace-6a97-4377-ae9a-3f619f71ae95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:09.385792 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1329/2616] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:09.394661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856a2ff-c2e8-4511-9c98-7b955fe8cb7c None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.394998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856a2ff-c2e8-4511-9c98-7b955fe8cb7c None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.395225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5856a2ff-c2e8-4511-9c98-7b955fe8cb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.403949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1288/2617] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 13:01:09.412494 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116092ad-4587-40b1-a165-c24a27abd208 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.412779 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116092ad-4587-40b1-a165-c24a27abd208 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.413048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116092ad-4587-40b1-a165-c24a27abd208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.417008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-5856a2ff-c2e8-4511-9c98-7b955fe8cb7c None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:09.418473 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5856a2ff-c2e8-4511-9c98-7b955fe8cb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:09.428412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1330/2618] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 13:01:09.432679 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856a2ff-c2e8-4511-9c98-7b955fe8cb7c None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:09.434523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-116092ad-4587-40b1-a165-c24a27abd208 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:09.435791 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-116092ad-4587-40b1-a165-c24a27abd208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:09.442753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5856a2ff-c2e8-4511-9c98-7b955fe8cb7c None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:09.444281 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1289/2619] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 13:01:09.448140 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682fc8a-9763-468f-8303-6e18f4ceece7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:09.448554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116092ad-4587-40b1-a165-c24a27abd208 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:09.448758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682fc8a-9763-468f-8303-6e18f4ceece7 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.449040 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682fc8a-9763-468f-8303-6e18f4ceece7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.449309 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682fc8a-9763-468f-8303-6e18f4ceece7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.450078 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682fc8a-9763-468f-8303-6e18f4ceece7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 13:01:09.450383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682fc8a-9763-468f-8303-6e18f4ceece7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:09.452089 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9682fc8a-9763-468f-8303-6e18f4ceece7 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:09.459469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116092ad-4587-40b1-a165-c24a27abd208 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:09.464486 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0393e6-828f-404b-8ec3-18f5b8510506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:09.464645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1331/2620] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 13:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 13:01:09.465161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0393e6-828f-404b-8ec3-18f5b8510506 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.465452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0393e6-828f-404b-8ec3-18f5b8510506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.465722 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0393e6-828f-404b-8ec3-18f5b8510506 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.466447 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0393e6-828f-404b-8ec3-18f5b8510506 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 13:01:09.466775 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0393e6-828f-404b-8ec3-18f5b8510506 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:09.468510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0393e6-828f-404b-8ec3-18f5b8510506 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:09.468908 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682fc8a-9763-468f-8303-6e18f4ceece7 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:09.480335 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682fc8a-9763-468f-8303-6e18f4ceece7 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:09.482655 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1290/2621] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 13:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 13:01:09.486707 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0393e6-828f-404b-8ec3-18f5b8510506 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:09.488686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079f310-1016-4484-b6a2-2c4b8da4039b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:09.489364 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079f310-1016-4484-b6a2-2c4b8da4039b admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.489640 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079f310-1016-4484-b6a2-2c4b8da4039b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.489907 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9079f310-1016-4484-b6a2-2c4b8da4039b admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.490606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079f310-1016-4484-b6a2-2c4b8da4039b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:09.492490 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9079f310-1016-4484-b6a2-2c4b8da4039b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:09.496698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0393e6-828f-404b-8ec3-18f5b8510506 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:09.501339 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ef41c-e4d8-4bdc-93ab-b0f5788ac289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:09.501938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ef41c-e4d8-4bdc-93ab-b0f5788ac289 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.502220 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ef41c-e4d8-4bdc-93ab-b0f5788ac289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.502521 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257ef41c-e4d8-4bdc-93ab-b0f5788ac289 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.503196 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ef41c-e4d8-4bdc-93ab-b0f5788ac289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:09.505067 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257ef41c-e4d8-4bdc-93ab-b0f5788ac289 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:09.510566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:09.520662 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1332/2622] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:01:09 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:01:09.525207 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079f310-1016-4484-b6a2-2c4b8da4039b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:09.539027 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1291/2623] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:01:09 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 13:01:09.546454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ef41c-e4d8-4bdc-93ab-b0f5788ac289 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:09.568970 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:09.594026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9079f310-1016-4484-b6a2-2c4b8da4039b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:09.609709 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257ef41c-e4d8-4bdc-93ab-b0f5788ac289 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:09.615131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736eba47-97cd-47cd-8cc2-b1121953dee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:09.615747 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736eba47-97cd-47cd-8cc2-b1121953dee2 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.616027 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736eba47-97cd-47cd-8cc2-b1121953dee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.616328 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736eba47-97cd-47cd-8cc2-b1121953dee2 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.616988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736eba47-97cd-47cd-8cc2-b1121953dee2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:09.618945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736eba47-97cd-47cd-8cc2-b1121953dee2 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:09.640521 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1292/2624] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:01:09 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:01:09.645292 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736eba47-97cd-47cd-8cc2-b1121953dee2 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:09.651925 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db68611-1713-4b6d-89a2-5474f50d9b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:09.652545 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db68611-1713-4b6d-89a2-5474f50d9b3f admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.652823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db68611-1713-4b6d-89a2-5474f50d9b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.653106 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db68611-1713-4b6d-89a2-5474f50d9b3f admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.653769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db68611-1713-4b6d-89a2-5474f50d9b3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:09.654709 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736eba47-97cd-47cd-8cc2-b1121953dee2 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:09.655978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db68611-1713-4b6d-89a2-5474f50d9b3f admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:09.660064 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48272566-2502-465f-943a-941d8a1f7d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:09.660654 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48272566-2502-465f-943a-941d8a1f7d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.660932 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48272566-2502-465f-943a-941d8a1f7d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.661207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48272566-2502-465f-943a-941d8a1f7d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.661880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48272566-2502-465f-943a-941d8a1f7d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:09.663618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48272566-2502-465f-943a-941d8a1f7d13 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:09.673814 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1293/2625] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 13:01:09.677455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1333/2626] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:01:09 2022] POST /identity/v3/users => generated 291 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 13:01:09.678075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48272566-2502-465f-943a-941d8a1f7d13 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:09.681206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db68611-1713-4b6d-89a2-5474f50d9b3f admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:09.688332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48272566-2502-465f-943a-941d8a1f7d13 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:09.691830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db68611-1713-4b6d-89a2-5474f50d9b3f admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:09.692971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b66720-5a5f-473d-991e-3b4381b0e001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:09.693577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b66720-5a5f-473d-991e-3b4381b0e001 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.693856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b66720-5a5f-473d-991e-3b4381b0e001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.694127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b66720-5a5f-473d-991e-3b4381b0e001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.694878 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b66720-5a5f-473d-991e-3b4381b0e001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:09.696457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b66720-5a5f-473d-991e-3b4381b0e001 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:09.696580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f75d9f-800f-4609-9bc8-3583f42223aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:09.697169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f75d9f-800f-4609-9bc8-3583f42223aa admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.697452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f75d9f-800f-4609-9bc8-3583f42223aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.698238 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f75d9f-800f-4609-9bc8-3583f42223aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.700262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f75d9f-800f-4609-9bc8-3583f42223aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:09.700959 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f75d9f-800f-4609-9bc8-3583f42223aa admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:09.707951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1294/2627] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:01:09.711971 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b66720-5a5f-473d-991e-3b4381b0e001 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:09.713891 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1334/2628] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:01:09.717617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f75d9f-800f-4609-9bc8-3583f42223aa admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:09.722729 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b66720-5a5f-473d-991e-3b4381b0e001 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:09.730642 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f75d9f-800f-4609-9bc8-3583f42223aa admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:09.731866 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab91c40-2448-49f5-8c99-64eca8715f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:09.732658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab91c40-2448-49f5-8c99-64eca8715f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.732982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab91c40-2448-49f5-8c99-64eca8715f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.733313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab91c40-2448-49f5-8c99-64eca8715f1c admin admin] PATH_INFO: `/v3/projects/4dc6dac8b9044792a15633af68931533/users/9245bd2079df4931846baaf7c3b54e55/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.735891 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d20f86-8e1f-466d-b0cb-8a55f8e99988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:09.736498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d20f86-8e1f-466d-b0cb-8a55f8e99988 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.736781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d20f86-8e1f-466d-b0cb-8a55f8e99988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.737569 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d20f86-8e1f-466d-b0cb-8a55f8e99988 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.737777 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d20f86-8e1f-466d-b0cb-8a55f8e99988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:09.739511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d20f86-8e1f-466d-b0cb-8a55f8e99988 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:09.752946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab91c40-2448-49f5-8c99-64eca8715f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dc6dac8b9044792a15633af68931533, user_id=9245bd2079df4931846baaf7c3b54e55, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:09.756828 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1335/2629] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:01:09.756968 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab91c40-2448-49f5-8c99-64eca8715f1c admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:09.757862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d20f86-8e1f-466d-b0cb-8a55f8e99988 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:09.771274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1295/2630] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:01:09 2022] PUT /identity/v3/projects/4dc6dac8b9044792a15633af68931533/users/9245bd2079df4931846baaf7c3b54e55/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 13:01:09.775144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d20f86-8e1f-466d-b0cb-8a55f8e99988 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:09.776207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5bd07-3c53-4597-9e77-0df309e32bdc None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.776494 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5bd07-3c53-4597-9e77-0df309e32bdc None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.776764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5bd07-3c53-4597-9e77-0df309e32bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.799070 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-97d5bd07-3c53-4597-9e77-0df309e32bdc None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:09.800458 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-97d5bd07-3c53-4597-9e77-0df309e32bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:09.847362 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4c640-aeef-4428-884e-5a86f2c12ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:09.847951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4c640-aeef-4428-884e-5a86f2c12ade admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.848230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4c640-aeef-4428-884e-5a86f2c12ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.848548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4c640-aeef-4428-884e-5a86f2c12ade admin admin] PATH_INFO: `/v3/projects/a3adb01c3232480ebf837645444732cc/users/b7bd8313a5da4febb4b91ef1107f387f/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.866470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4c640-aeef-4428-884e-5a86f2c12ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3adb01c3232480ebf837645444732cc, user_id=b7bd8313a5da4febb4b91ef1107f387f, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:09.882335 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4c640-aeef-4428-884e-5a86f2c12ade admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:09.884201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1336/2631] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:01:09 2022] PUT /identity/v3/projects/a3adb01c3232480ebf837645444732cc/users/b7bd8313a5da4febb4b91ef1107f387f/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 13:01:09.895366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1296/2632] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 13:01:09.895542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd366f86-d32e-4cab-a5ad-63b693274db5 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:09.895815 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd366f86-d32e-4cab-a5ad-63b693274db5 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:09.896150 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd366f86-d32e-4cab-a5ad-63b693274db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:09.904848 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5bd07-3c53-4597-9e77-0df309e32bdc None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:09.906246 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-97d5bd07-3c53-4597-9e77-0df309e32bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:09.924168 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-cd366f86-d32e-4cab-a5ad-63b693274db5 None None] MFA Rules not processed for user `b7bd8313a5da4febb4b91ef1107f387f`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:09.925354 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-cd366f86-d32e-4cab-a5ad-63b693274db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:09.940818 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:09.944724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:09.966026 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5bd07-3c53-4597-9e77-0df309e32bdc None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:09.990274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1337/2633] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 13:01:09.994426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd366f86-d32e-4cab-a5ad-63b693274db5 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:10.017955 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01554d-cb9c-4fa3-9383-ac5d850911ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:10.018628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01554d-cb9c-4fa3-9383-ac5d850911ef service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:10.018920 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01554d-cb9c-4fa3-9383-ac5d850911ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:10.019193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01554d-cb9c-4fa3-9383-ac5d850911ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:10.060869 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd366f86-d32e-4cab-a5ad-63b693274db5 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:10.066328 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1ec83-320a-44ba-b14f-daa5981a616d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:10.067134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1ec83-320a-44ba-b14f-daa5981a616d admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:10.067413 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1ec83-320a-44ba-b14f-daa5981a616d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:10.068054 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1ec83-320a-44ba-b14f-daa5981a616d admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:10.068546 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1ec83-320a-44ba-b14f-daa5981a616d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:10.070601 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1ec83-320a-44ba-b14f-daa5981a616d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:10.092247 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01554d-cb9c-4fa3-9383-ac5d850911ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:10.093705 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01554d-cb9c-4fa3-9383-ac5d850911ef service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:10.103607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1338/2634] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:01:09 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 13:01:10.105705 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1297/2635] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:01:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 13:01:10.107498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1ec83-320a-44ba-b14f-daa5981a616d admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:10.200057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1ec83-320a-44ba-b14f-daa5981a616d admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:10.207647 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e25b9-e539-4a90-98f2-d7253c9447c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:10.208358 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e25b9-e539-4a90-98f2-d7253c9447c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:10.208636 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e25b9-e539-4a90-98f2-d7253c9447c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:10.208967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e25b9-e539-4a90-98f2-d7253c9447c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:10.210028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e25b9-e539-4a90-98f2-d7253c9447c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:10.212101 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e25b9-e539-4a90-98f2-d7253c9447c4 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:10.238674 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1339/2636] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:01:10 2022] POST /identity/v3/users => generated 298 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:01:10.240418 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01554d-cb9c-4fa3-9383-ac5d850911ef service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:10.257932 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01554d-cb9c-4fa3-9383-ac5d850911ef service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:10.264964 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68acfc-33e5-43d0-891a-0f266377a3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:10.265873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68acfc-33e5-43d0-891a-0f266377a3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:10.266919 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68acfc-33e5-43d0-891a-0f266377a3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:10.267424 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68acfc-33e5-43d0-891a-0f266377a3d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:10.268700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68acfc-33e5-43d0-891a-0f266377a3d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:10.271896 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68acfc-33e5-43d0-891a-0f266377a3d9 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:10.294117 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1298/2637] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:01:10.295664 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e25b9-e539-4a90-98f2-d7253c9447c4 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:10.306640 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e25b9-e539-4a90-98f2-d7253c9447c4 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:10.311807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840477f-5391-49f0-89f9-ffd708daa435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:10.312571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840477f-5391-49f0-89f9-ffd708daa435 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:10.312865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840477f-5391-49f0-89f9-ffd708daa435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:10.313736 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840477f-5391-49f0-89f9-ffd708daa435 admin admin] PATH_INFO: `/v3/projects/d49723db69ce48769e0f7530ae1e0f6b/users/86bb3e5064684620a6632150cc8095a7/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:10.338027 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840477f-5391-49f0-89f9-ffd708daa435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d49723db69ce48769e0f7530ae1e0f6b, user_id=86bb3e5064684620a6632150cc8095a7, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:10.341009 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840477f-5391-49f0-89f9-ffd708daa435 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:10.357875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1340/2638] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:01:10 2022] PUT /identity/v3/projects/d49723db69ce48769e0f7530ae1e0f6b/users/86bb3e5064684620a6632150cc8095a7/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 13:01:10.361649 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68acfc-33e5-43d0-891a-0f266377a3d9 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:10.372105 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68acfc-33e5-43d0-891a-0f266377a3d9 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:10.435013 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5b9be-723d-492a-bc7b-3ce5b931fd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:10.435640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5b9be-723d-492a-bc7b-3ce5b931fd8e admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:10.435920 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5b9be-723d-492a-bc7b-3ce5b931fd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:10.436187 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5b9be-723d-492a-bc7b-3ce5b931fd8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:10.437060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5b9be-723d-492a-bc7b-3ce5b931fd8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:10.439444 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5b9be-723d-492a-bc7b-3ce5b931fd8e admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:10.455554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1299/2639] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 13:01:10.459488 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840477f-5391-49f0-89f9-ffd708daa435 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:10.471013 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840477f-5391-49f0-89f9-ffd708daa435 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:10.476749 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bfdee-1a44-4c35-92cd-a37c1d052a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:10.477842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bfdee-1a44-4c35-92cd-a37c1d052a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:10.478237 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bfdee-1a44-4c35-92cd-a37c1d052a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:10.478713 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bfdee-1a44-4c35-92cd-a37c1d052a0b admin admin] PATH_INFO: `/v3/projects/d49723db69ce48769e0f7530ae1e0f6b/users/86bb3e5064684620a6632150cc8095a7/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:10.482667 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bfdee-1a44-4c35-92cd-a37c1d052a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d49723db69ce48769e0f7530ae1e0f6b, user_id=86bb3e5064684620a6632150cc8095a7, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:10.488292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bfdee-1a44-4c35-92cd-a37c1d052a0b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:10.496248 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:10.502530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1341/2640] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:01:10 2022] PUT /identity/v3/projects/d49723db69ce48769e0f7530ae1e0f6b/users/86bb3e5064684620a6632150cc8095a7/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:10.513241 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f873dcc-457d-477f-9d26-58728cf4c77e None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:10.513440 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f873dcc-457d-477f-9d26-58728cf4c77e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:10.513765 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f873dcc-457d-477f-9d26-58728cf4c77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:10.543298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-8f873dcc-457d-477f-9d26-58728cf4c77e None None] MFA Rules not processed for user `86bb3e5064684620a6632150cc8095a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:10.544677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8f873dcc-457d-477f-9d26-58728cf4c77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:10.558764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:10.633348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1300/2641] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:10.645995 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bfdee-1a44-4c35-92cd-a37c1d052a0b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:10.647937 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c80bfdee-1a44-4c35-92cd-a37c1d052a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:10.730176 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bfdee-1a44-4c35-92cd-a37c1d052a0b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:10.733885 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0f647-1244-41a4-947c-bced9b7f1109 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:10.734526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0f647-1244-41a4-947c-bced9b7f1109 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:10.735329 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0f647-1244-41a4-947c-bced9b7f1109 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:10.735603 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0f647-1244-41a4-947c-bced9b7f1109 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:10.745107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0f647-1244-41a4-947c-bced9b7f1109 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:10.746543 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0f647-1244-41a4-947c-bced9b7f1109 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:10.756600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1342/2642] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:01:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:01:10.901184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f873dcc-457d-477f-9d26-58728cf4c77e None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:10.913053 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f873dcc-457d-477f-9d26-58728cf4c77e None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:10.917184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082158b-430c-417f-ab7e-362d4b5e9f85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:10.917816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082158b-430c-417f-ab7e-362d4b5e9f85 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:10.918508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082158b-430c-417f-ab7e-362d4b5e9f85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:10.918508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082158b-430c-417f-ab7e-362d4b5e9f85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:10.999304 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082158b-430c-417f-ab7e-362d4b5e9f85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:11.001990 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082158b-430c-417f-ab7e-362d4b5e9f85 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:11.017711 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1301/2643] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:01:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:01:11.540346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0f647-1244-41a4-947c-bced9b7f1109 service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:11.618264 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0f647-1244-41a4-947c-bced9b7f1109 service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:11.624302 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcd3e9-0f1f-467f-9e88-e711deb97688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:11.625108 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcd3e9-0f1f-467f-9e88-e711deb97688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:11.625919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcd3e9-0f1f-467f-9e88-e711deb97688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:11.626178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcd3e9-0f1f-467f-9e88-e711deb97688 admin admin] PATH_INFO: `/v3/users/b7bd8313a5da4febb4b91ef1107f387f` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:11.634245 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbcd3e9-0f1f-467f-9e88-e711deb97688 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7bd8313a5da4febb4b91ef1107f387f)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:11.636257 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbcd3e9-0f1f-467f-9e88-e711deb97688 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:11.701700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bfbcd3e9-0f1f-467f-9e88-e711deb97688 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7bd8313a5da4febb4b91ef1107f387f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:11.727092 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bfbcd3e9-0f1f-467f-9e88-e711deb97688 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7bd8313a5da4febb4b91ef1107f387f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:11.747802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bfbcd3e9-0f1f-467f-9e88-e711deb97688 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7bd8313a5da4febb4b91ef1107f387f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:11.748559 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bfbcd3e9-0f1f-467f-9e88-e711deb97688 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7bd8313a5da4febb4b91ef1107f387f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:11.749238 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-bfbcd3e9-0f1f-467f-9e88-e711deb97688 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7bd8313a5da4febb4b91ef1107f387f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:11.764168 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1343/2644] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:01:11 2022] DELETE /identity/v3/users/b7bd8313a5da4febb4b91ef1107f387f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:11.772313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962644b-a0db-4a26-9f5a-d738d838cb18 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:11.772810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962644b-a0db-4a26-9f5a-d738d838cb18 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:11.773046 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962644b-a0db-4a26-9f5a-d738d838cb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:11.796011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-d962644b-a0db-4a26-9f5a-d738d838cb18 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:11.797338 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d962644b-a0db-4a26-9f5a-d738d838cb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:11.868816 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1302/2645] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:11.880187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcd3e9-0f1f-467f-9e88-e711deb97688 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:11.881769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-bfbcd3e9-0f1f-467f-9e88-e711deb97688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:11.915160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:11.948251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcd3e9-0f1f-467f-9e88-e711deb97688 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:11.951889 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae271c1-6b1d-479b-ac54-2afbbf052bbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:11.952444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae271c1-6b1d-479b-ac54-2afbbf052bbd service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:11.952724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae271c1-6b1d-479b-ac54-2afbbf052bbd service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:11.953022 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae271c1-6b1d-479b-ac54-2afbbf052bbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:11.962088 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae271c1-6b1d-479b-ac54-2afbbf052bbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:11.963552 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae271c1-6b1d-479b-ac54-2afbbf052bbd service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:11.972952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1344/2646] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:01:12.296869 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962644b-a0db-4a26-9f5a-d738d838cb18 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:12.298693 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d962644b-a0db-4a26-9f5a-d738d838cb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:12.335853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962644b-a0db-4a26-9f5a-d738d838cb18 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:12.341918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:12.342623 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:12.342907 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:12.343320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb admin admin] PATH_INFO: `/v3/projects/a3adb01c3232480ebf837645444732cc` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:12.344999 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3adb01c3232480ebf837645444732cc)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:12.348243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:12.405856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:12.416730 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3adb01c3232480ebf837645444732cc', 'request_id': 'req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:12.430559 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3adb01c3232480ebf837645444732cc', 'request_id': 'req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:12.439801 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3adb01c3232480ebf837645444732cc', 'request_id': 'req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:12.453386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb admin admin] The token cache is being invalidate because project a3adb01c3232480ebf837645444732cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:01:12.454822 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:12.456020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:12.457372 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1303/2647] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:01:12 2022] DELETE /identity/v3/projects/a3adb01c3232480ebf837645444732cc => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:12.462218 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae271c1-6b1d-479b-ac54-2afbbf052bbd service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:12.478164 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae271c1-6b1d-479b-ac54-2afbbf052bbd service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:12.612623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea9ad76-dfa3-4801-a525-b6ab96f57e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:12.613398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9ad76-dfa3-4801-a525-b6ab96f57e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:12.613785 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9ad76-dfa3-4801-a525-b6ab96f57e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:12.614165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9ad76-dfa3-4801-a525-b6ab96f57e01 admin admin] PATH_INFO: `/v3/users/86bb3e5064684620a6632150cc8095a7` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:12.616805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea9ad76-dfa3-4801-a525-b6ab96f57e01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86bb3e5064684620a6632150cc8095a7)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:12.618927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea9ad76-dfa3-4801-a525-b6ab96f57e01 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:12.755077 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-4ea9ad76-dfa3-4801-a525-b6ab96f57e01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86bb3e5064684620a6632150cc8095a7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:12.763805 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-4ea9ad76-dfa3-4801-a525-b6ab96f57e01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86bb3e5064684620a6632150cc8095a7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:12.778645 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-4ea9ad76-dfa3-4801-a525-b6ab96f57e01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86bb3e5064684620a6632150cc8095a7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:12.779769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-4ea9ad76-dfa3-4801-a525-b6ab96f57e01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86bb3e5064684620a6632150cc8095a7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:12.780487 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-4ea9ad76-dfa3-4801-a525-b6ab96f57e01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86bb3e5064684620a6632150cc8095a7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:12.809790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1345/2648] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:01:12 2022] DELETE /identity/v3/users/86bb3e5064684620a6632150cc8095a7 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:13.158677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:13.160209 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:13.180597 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf4f9ba-1a13-4f12-8165-818e3b1ff9fb admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:13.253544 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5b998-e930-4721-96d4-2d40894a15ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:13.254976 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5b998-e930-4721-96d4-2d40894a15ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:13.255498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5b998-e930-4721-96d4-2d40894a15ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:13.255880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5b998-e930-4721-96d4-2d40894a15ff admin admin] PATH_INFO: `/v3/projects/d49723db69ce48769e0f7530ae1e0f6b` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:13.257495 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5b998-e930-4721-96d4-2d40894a15ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=d49723db69ce48769e0f7530ae1e0f6b)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:13.259825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5b998-e930-4721-96d4-2d40894a15ff admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:13.334008 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-63a5b998-e930-4721-96d4-2d40894a15ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd49723db69ce48769e0f7530ae1e0f6b', 'request_id': 'req-63a5b998-e930-4721-96d4-2d40894a15ff'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:13.345967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-63a5b998-e930-4721-96d4-2d40894a15ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd49723db69ce48769e0f7530ae1e0f6b', 'request_id': 'req-63a5b998-e930-4721-96d4-2d40894a15ff'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:13.354023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-63a5b998-e930-4721-96d4-2d40894a15ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd49723db69ce48769e0f7530ae1e0f6b', 'request_id': 'req-63a5b998-e930-4721-96d4-2d40894a15ff'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:13.367407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-63a5b998-e930-4721-96d4-2d40894a15ff admin admin] The token cache is being invalidate because project d49723db69ce48769e0f7530ae1e0f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:01:13.367924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-63a5b998-e930-4721-96d4-2d40894a15ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:13.368644 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-63a5b998-e930-4721-96d4-2d40894a15ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:13.370438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1304/2649] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:01:13 2022] DELETE /identity/v3/projects/d49723db69ce48769e0f7530ae1e0f6b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:13.375153 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70167d2e-fe5a-4843-9e32-c6123e50e191 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:13.375575 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70167d2e-fe5a-4843-9e32-c6123e50e191 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:13.376588 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70167d2e-fe5a-4843-9e32-c6123e50e191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:13.402363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-70167d2e-fe5a-4843-9e32-c6123e50e191 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:13.403473 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-70167d2e-fe5a-4843-9e32-c6123e50e191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:13.456276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:13.483571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1346/2650] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:13.491698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eefb9b-1a7b-4e88-95f7-2658a612b314 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:13.492155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eefb9b-1a7b-4e88-95f7-2658a612b314 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:13.492609 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eefb9b-1a7b-4e88-95f7-2658a612b314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:13.519055 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-80eefb9b-1a7b-4e88-95f7-2658a612b314 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:13.520615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-80eefb9b-1a7b-4e88-95f7-2658a612b314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:13.529084 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1305/2651] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:13.535237 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70167d2e-fe5a-4843-9e32-c6123e50e191 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:13.548152 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70167d2e-fe5a-4843-9e32-c6123e50e191 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:13.556067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c9a3f-365f-4942-9d02-7c1d93edbbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:13.557318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c9a3f-365f-4942-9d02-7c1d93edbbb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:13.557318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c9a3f-365f-4942-9d02-7c1d93edbbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:13.557318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c9a3f-365f-4942-9d02-7c1d93edbbb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:13.557909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c9a3f-365f-4942-9d02-7c1d93edbbb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 13:01:13.558209 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c9a3f-365f-4942-9d02-7c1d93edbbb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:13.559841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c9a3f-365f-4942-9d02-7c1d93edbbb8 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:13.572263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1347/2652] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 13:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 13:01:13.576685 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eefb9b-1a7b-4e88-95f7-2658a612b314 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:13.588664 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eefb9b-1a7b-4e88-95f7-2658a612b314 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:13.594754 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21603e39-49a8-429a-a087-c91ac626f057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:13.595508 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21603e39-49a8-429a-a087-c91ac626f057 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:13.595920 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21603e39-49a8-429a-a087-c91ac626f057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:13.596284 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21603e39-49a8-429a-a087-c91ac626f057 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:13.597031 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21603e39-49a8-429a-a087-c91ac626f057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:13.599051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21603e39-49a8-429a-a087-c91ac626f057 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:13.636286 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1306/2653] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:01:13 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:01:13.637010 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c9a3f-365f-4942-9d02-7c1d93edbbb8 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:13.649181 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c9a3f-365f-4942-9d02-7c1d93edbbb8 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:13.720141 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532e384-c784-4ff9-b6e5-b811fda1e8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:13.720835 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532e384-c784-4ff9-b6e5-b811fda1e8a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:13.721251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532e384-c784-4ff9-b6e5-b811fda1e8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:13.721622 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532e384-c784-4ff9-b6e5-b811fda1e8a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:13.722381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3532e384-c784-4ff9-b6e5-b811fda1e8a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:13.724620 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3532e384-c784-4ff9-b6e5-b811fda1e8a6 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:13.755043 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1348/2654] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:01:13 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 13:01:13.759312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21603e39-49a8-429a-a087-c91ac626f057 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:13.779714 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21603e39-49a8-429a-a087-c91ac626f057 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:13.790677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcc014-00b9-4c56-97a2-7f6a4c1af6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:13.792020 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcc014-00b9-4c56-97a2-7f6a4c1af6e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:13.792377 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcc014-00b9-4c56-97a2-7f6a4c1af6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:13.792654 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fcc014-00b9-4c56-97a2-7f6a4c1af6e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:13.793371 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcc014-00b9-4c56-97a2-7f6a4c1af6e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:13.795572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fcc014-00b9-4c56-97a2-7f6a4c1af6e4 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:13.854178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1307/2655] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 13:01:13.858585 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532e384-c784-4ff9-b6e5-b811fda1e8a6 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:13.884093 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532e384-c784-4ff9-b6e5-b811fda1e8a6 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:13.899622 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c0758-ccd8-4d9b-b60d-d7d291a13671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:13.900478 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c0758-ccd8-4d9b-b60d-d7d291a13671 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:13.900996 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c0758-ccd8-4d9b-b60d-d7d291a13671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:13.901433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c0758-ccd8-4d9b-b60d-d7d291a13671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:13.902347 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c0758-ccd8-4d9b-b60d-d7d291a13671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:13.907762 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c0758-ccd8-4d9b-b60d-d7d291a13671 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:13.920981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1349/2656] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:13 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:01:13.926117 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcc014-00b9-4c56-97a2-7f6a4c1af6e4 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:13.937936 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fcc014-00b9-4c56-97a2-7f6a4c1af6e4 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:13.945741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489ca44-8ba0-479c-b91f-c4c57d343375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:13.946657 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489ca44-8ba0-479c-b91f-c4c57d343375 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:13.947089 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489ca44-8ba0-479c-b91f-c4c57d343375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:13.947650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489ca44-8ba0-479c-b91f-c4c57d343375 admin admin] PATH_INFO: `/v3/projects/401c7126a6f24240bbcd50e86a136d88/users/8926a23cf38d483fb31901af271f76b7/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:13.984304 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489ca44-8ba0-479c-b91f-c4c57d343375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=401c7126a6f24240bbcd50e86a136d88, user_id=8926a23cf38d483fb31901af271f76b7, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:13.992161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489ca44-8ba0-479c-b91f-c4c57d343375 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:14.011912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:14.040914 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1308/2657] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:01:13 2022] PUT /identity/v3/projects/401c7126a6f24240bbcd50e86a136d88/users/8926a23cf38d483fb31901af271f76b7/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:14.045890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f34b7-b2f0-4c93-8550-07ab06fa8f26 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:14.046417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f34b7-b2f0-4c93-8550-07ab06fa8f26 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:14.046720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f34b7-b2f0-4c93-8550-07ab06fa8f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:14.068827 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-dc0f34b7-b2f0-4c93-8550-07ab06fa8f26 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:14.070464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-dc0f34b7-b2f0-4c93-8550-07ab06fa8f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:14.155565 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1350/2658] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:14.162562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489ca44-8ba0-479c-b91f-c4c57d343375 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:14.164443 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8489ca44-8ba0-479c-b91f-c4c57d343375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:14.231464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489ca44-8ba0-479c-b91f-c4c57d343375 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:14.235996 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab82460-3a76-4ee4-9b3e-670b71e555fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:14.236860 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab82460-3a76-4ee4-9b3e-670b71e555fe service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:14.237134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab82460-3a76-4ee4-9b3e-670b71e555fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:14.237623 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab82460-3a76-4ee4-9b3e-670b71e555fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:14.248447 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab82460-3a76-4ee4-9b3e-670b71e555fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:14.250083 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab82460-3a76-4ee4-9b3e-670b71e555fe service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:14.261335 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1309/2659] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:01:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:01:15.054321 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e22aa-54d8-4aa2-a57b-fbe6f9f96c21 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:15.055037 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e22aa-54d8-4aa2-a57b-fbe6f9f96c21 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:15.055037 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e22aa-54d8-4aa2-a57b-fbe6f9f96c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:15.097353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-316e22aa-54d8-4aa2-a57b-fbe6f9f96c21 None None] MFA Rules not processed for user `9245bd2079df4931846baaf7c3b54e55`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:15.098854 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-316e22aa-54d8-4aa2-a57b-fbe6f9f96c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:15.166841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:15.205795 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfbcac-a8f5-4a24-a822-8c36c3105b95 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:15.206337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfbcac-a8f5-4a24-a822-8c36c3105b95 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:15.207103 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bfbcac-a8f5-4a24-a822-8c36c3105b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:15.209940 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1351/2660] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:15.211124 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e22aa-54d8-4aa2-a57b-fbe6f9f96c21 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:15.212658 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-316e22aa-54d8-4aa2-a57b-fbe6f9f96c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:15.233007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e22aa-54d8-4aa2-a57b-fbe6f9f96c21 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:15.242117 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56a0a0-0d7f-41ef-bb86-8fb48daebe80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:15.243074 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56a0a0-0d7f-41ef-bb86-8fb48daebe80 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:15.243560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56a0a0-0d7f-41ef-bb86-8fb48daebe80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:15.244006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd56a0a0-0d7f-41ef-bb86-8fb48daebe80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:15.244261 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-b6bfbcac-a8f5-4a24-a822-8c36c3105b95 None None] MFA Rules not processed for user `8926a23cf38d483fb31901af271f76b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:15.245129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56a0a0-0d7f-41ef-bb86-8fb48daebe80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:15.245673 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-b6bfbcac-a8f5-4a24-a822-8c36c3105b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:15.252662 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd56a0a0-0d7f-41ef-bb86-8fb48daebe80 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:15.287563 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1352/2661] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:01:15 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:01:15.297120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56a0a0-0d7f-41ef-bb86-8fb48daebe80 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:15.369401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1310/2662] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:15.378167 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfbcac-a8f5-4a24-a822-8c36c3105b95 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:15.400703 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bfbcac-a8f5-4a24-a822-8c36c3105b95 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:15.404627 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd56a0a0-0d7f-41ef-bb86-8fb48daebe80 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:15.411712 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0779fd2-06cb-444d-a09f-e641b3d35b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:15.412553 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0779fd2-06cb-444d-a09f-e641b3d35b77 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:15.412991 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0779fd2-06cb-444d-a09f-e641b3d35b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:15.413503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0779fd2-06cb-444d-a09f-e641b3d35b77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:15.414589 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0779fd2-06cb-444d-a09f-e641b3d35b77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:15.416650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0779fd2-06cb-444d-a09f-e641b3d35b77 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:15.422167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a73c7-72bf-4d03-a70a-519bf18812ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:15.423413 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a73c7-72bf-4d03-a70a-519bf18812ae admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:15.424341 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a73c7-72bf-4d03-a70a-519bf18812ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:15.424341 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11a73c7-72bf-4d03-a70a-519bf18812ae admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:15.424900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a73c7-72bf-4d03-a70a-519bf18812ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:15.427026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11a73c7-72bf-4d03-a70a-519bf18812ae admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:15.468300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1311/2663] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:01:15 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:01:15.472924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0779fd2-06cb-444d-a09f-e641b3d35b77 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:15.473080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1353/2664] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:01:15 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:01:15.477674 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a73c7-72bf-4d03-a70a-519bf18812ae admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:15.495766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11a73c7-72bf-4d03-a70a-519bf18812ae admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:15.595807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b1e34-741c-4c06-ad17-e6ffedc457ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:15.597347 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b1e34-741c-4c06-ad17-e6ffedc457ad admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:15.597781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b1e34-741c-4c06-ad17-e6ffedc457ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:15.598204 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b1e34-741c-4c06-ad17-e6ffedc457ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:15.603327 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b1e34-741c-4c06-ad17-e6ffedc457ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:15.605530 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b1e34-741c-4c06-ad17-e6ffedc457ad admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:15.613195 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0779fd2-06cb-444d-a09f-e641b3d35b77 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:15.618624 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3457fe-a937-4e3b-a9f0-628d6a315891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:15.619425 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3457fe-a937-4e3b-a9f0-628d6a315891 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:15.619808 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3457fe-a937-4e3b-a9f0-628d6a315891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:15.620199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3457fe-a937-4e3b-a9f0-628d6a315891 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:15.620816 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1354/2665] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:01:15.624292 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3457fe-a937-4e3b-a9f0-628d6a315891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:15.626475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b1e34-741c-4c06-ad17-e6ffedc457ad admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:15.626664 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3457fe-a937-4e3b-a9f0-628d6a315891 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:15.638700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:15.638936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b1e34-741c-4c06-ad17-e6ffedc457ad admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:15.646006 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbbc82-0df3-47b7-a2a4-28cf5eeee6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:15.646724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbbc82-0df3-47b7-a2a4-28cf5eeee6ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:15.647008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbbc82-0df3-47b7-a2a4-28cf5eeee6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:15.647339 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbbc82-0df3-47b7-a2a4-28cf5eeee6ed admin admin] PATH_INFO: `/v3/projects/c95d59c048e54d8d805c5f82a5aa78d7/users/e9fa409c38ce410aa46d743e6bf017b1/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:15.661048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1312/2666] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:01:15 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:01:15.662811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3457fe-a937-4e3b-a9f0-628d6a315891 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:15.665560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbbc82-0df3-47b7-a2a4-28cf5eeee6ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95d59c048e54d8d805c5f82a5aa78d7, user_id=e9fa409c38ce410aa46d743e6bf017b1, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:15.669003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbbc82-0df3-47b7-a2a4-28cf5eeee6ed admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:15.686060 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3457fe-a937-4e3b-a9f0-628d6a315891 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:15.690567 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1355/2667] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:01:15 2022] PUT /identity/v3/projects/c95d59c048e54d8d805c5f82a5aa78d7/users/e9fa409c38ce410aa46d743e6bf017b1/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 13:01:15.694647 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbbc82-0df3-47b7-a2a4-28cf5eeee6ed admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:15.724305 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:15.761490 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebc5c0-8545-4d0e-afac-7d3d41f9d70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:15.762303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebc5c0-8545-4d0e-afac-7d3d41f9d70a admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:15.762727 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebc5c0-8545-4d0e-afac-7d3d41f9d70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:15.763095 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebc5c0-8545-4d0e-afac-7d3d41f9d70a admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:15.763892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebc5c0-8545-4d0e-afac-7d3d41f9d70a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:15.765588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ebc5c0-8545-4d0e-afac-7d3d41f9d70a admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:15.779855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1313/2668] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:01:15.780911 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebc5c0-8545-4d0e-afac-7d3d41f9d70a admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:15.788100 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbbc82-0df3-47b7-a2a4-28cf5eeee6ed admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:15.795163 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a57f96-f147-4e4c-8307-8c7737b37d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:15.795462 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebc5c0-8545-4d0e-afac-7d3d41f9d70a admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:15.795868 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a57f96-f147-4e4c-8307-8c7737b37d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:15.796159 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a57f96-f147-4e4c-8307-8c7737b37d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:15.796440 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a57f96-f147-4e4c-8307-8c7737b37d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:15.797335 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a57f96-f147-4e4c-8307-8c7737b37d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:15.799297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a57f96-f147-4e4c-8307-8c7737b37d61 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:15.801343 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16d671-279c-4de2-922b-515078db54bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:15.802983 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16d671-279c-4de2-922b-515078db54bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:15.807409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16d671-279c-4de2-922b-515078db54bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:15.807734 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16d671-279c-4de2-922b-515078db54bd admin admin] PATH_INFO: `/v3/projects/fe47cfe53a94443889010f625d1e4000/users/cb5bc84774d444f89ae3dbc5ab0f6a4f/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:15.813514 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1356/2669] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 13:01:15.818476 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a57f96-f147-4e4c-8307-8c7737b37d61 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:15.831906 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16d671-279c-4de2-922b-515078db54bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe47cfe53a94443889010f625d1e4000, user_id=cb5bc84774d444f89ae3dbc5ab0f6a4f, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:15.832155 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a57f96-f147-4e4c-8307-8c7737b37d61 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:15.834617 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16d671-279c-4de2-922b-515078db54bd admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:15.837777 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e01d54-e886-4453-92a7-3f09d5673371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:15.838636 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e01d54-e886-4453-92a7-3f09d5673371 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:15.839025 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e01d54-e886-4453-92a7-3f09d5673371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:15.839495 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e01d54-e886-4453-92a7-3f09d5673371 admin admin] PATH_INFO: `/v3/projects/c95d59c048e54d8d805c5f82a5aa78d7/users/e9fa409c38ce410aa46d743e6bf017b1/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:15.841924 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e01d54-e886-4453-92a7-3f09d5673371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95d59c048e54d8d805c5f82a5aa78d7, user_id=e9fa409c38ce410aa46d743e6bf017b1, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:15.846543 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e01d54-e886-4453-92a7-3f09d5673371 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:15.866041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1357/2670] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:01:15 2022] PUT /identity/v3/projects/c95d59c048e54d8d805c5f82a5aa78d7/users/e9fa409c38ce410aa46d743e6bf017b1/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 13:01:15.866250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1314/2671] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:01:15 2022] PUT /identity/v3/projects/fe47cfe53a94443889010f625d1e4000/users/cb5bc84774d444f89ae3dbc5ab0f6a4f/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:15.870518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e01d54-e886-4453-92a7-3f09d5673371 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:15.954426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e01d54-e886-4453-92a7-3f09d5673371 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:15.959313 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a536c-54d6-46cf-b035-09b3b5c2a618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:15.959990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a536c-54d6-46cf-b035-09b3b5c2a618 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:15.960282 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a536c-54d6-46cf-b035-09b3b5c2a618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:15.960560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a536c-54d6-46cf-b035-09b3b5c2a618 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:15.961265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a536c-54d6-46cf-b035-09b3b5c2a618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:15.963196 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a536c-54d6-46cf-b035-09b3b5c2a618 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:15.975056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1358/2672] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:01:15.979258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16d671-279c-4de2-922b-515078db54bd admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:16.051479 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16d671-279c-4de2-922b-515078db54bd admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:16.056223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6007b6-0f4d-47a8-8f46-c472b5060058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:16.056900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6007b6-0f4d-47a8-8f46-c472b5060058 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:16.057179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6007b6-0f4d-47a8-8f46-c472b5060058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:16.057449 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6007b6-0f4d-47a8-8f46-c472b5060058 admin admin] PATH_INFO: `/v3/projects/fe47cfe53a94443889010f625d1e4000/users/cb5bc84774d444f89ae3dbc5ab0f6a4f/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:16.059493 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6007b6-0f4d-47a8-8f46-c472b5060058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe47cfe53a94443889010f625d1e4000, user_id=cb5bc84774d444f89ae3dbc5ab0f6a4f, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:16.062171 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6007b6-0f4d-47a8-8f46-c472b5060058 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:16.075079 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1315/2673] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:01:15 2022] PUT /identity/v3/projects/fe47cfe53a94443889010f625d1e4000/users/cb5bc84774d444f89ae3dbc5ab0f6a4f/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:16.994468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b62b3-be22-4d80-bfcf-6edc20f22118 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:16.995033 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b62b3-be22-4d80-bfcf-6edc20f22118 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:16.995151 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b62b3-be22-4d80-bfcf-6edc20f22118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:17.049143 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-1d2b62b3-be22-4d80-bfcf-6edc20f22118 None None] MFA Rules not processed for user `cb5bc84774d444f89ae3dbc5ab0f6a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:17.050573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1d2b62b3-be22-4d80-bfcf-6edc20f22118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:17.215096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1359/2674] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:17.228908 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6007b6-0f4d-47a8-8f46-c472b5060058 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:17.231389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-0e6007b6-0f4d-47a8-8f46-c472b5060058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:17.370831 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6007b6-0f4d-47a8-8f46-c472b5060058 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:17.379363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a706fe8-bf56-4193-8d02-2bb1332b30c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:17.380008 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a706fe8-bf56-4193-8d02-2bb1332b30c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:17.380289 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a706fe8-bf56-4193-8d02-2bb1332b30c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:17.380702 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a706fe8-bf56-4193-8d02-2bb1332b30c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:17.395611 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:17.483909 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a706fe8-bf56-4193-8d02-2bb1332b30c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:17.486639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a706fe8-bf56-4193-8d02-2bb1332b30c2 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:17.497459 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1316/2675] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:01:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 03 13:01:19.089292 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b62b3-be22-4d80-bfcf-6edc20f22118 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:19.091416 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1d2b62b3-be22-4d80-bfcf-6edc20f22118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:19.110841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b62b3-be22-4d80-bfcf-6edc20f22118 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:19.116744 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce5821-6bde-40fc-b4c1-2b100adbf9d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:19.117657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce5821-6bde-40fc-b4c1-2b100adbf9d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:19.118714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce5821-6bde-40fc-b4c1-2b100adbf9d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:19.119157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce5821-6bde-40fc-b4c1-2b100adbf9d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:19.138422 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce5821-6bde-40fc-b4c1-2b100adbf9d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:19.139945 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce5821-6bde-40fc-b4c1-2b100adbf9d1 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:19.151973 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1360/2676] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:01:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:01:21.268983 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a4cd3-8ad2-4e45-8178-87aa24034359 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:21.269703 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a4cd3-8ad2-4e45-8178-87aa24034359 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:21.270587 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a4cd3-8ad2-4e45-8178-87aa24034359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:21.308902 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-5f6a4cd3-8ad2-4e45-8178-87aa24034359 None None] MFA Rules not processed for user `e9fa409c38ce410aa46d743e6bf017b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:21.310111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-5f6a4cd3-8ad2-4e45-8178-87aa24034359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:21.395854 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1317/2677] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:21.410651 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce5821-6bde-40fc-b4c1-2b100adbf9d1 service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:21.412211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e1ce5821-6bde-40fc-b4c1-2b100adbf9d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:21.426897 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:21.522683 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce5821-6bde-40fc-b4c1-2b100adbf9d1 service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:21.528320 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8dfab4-b4b2-43ec-82d2-1698f0f4adcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:21.528943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dfab4-b4b2-43ec-82d2-1698f0f4adcf service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:21.529226 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dfab4-b4b2-43ec-82d2-1698f0f4adcf service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:21.529504 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dfab4-b4b2-43ec-82d2-1698f0f4adcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:21.587057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8dfab4-b4b2-43ec-82d2-1698f0f4adcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:21.588658 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8dfab4-b4b2-43ec-82d2-1698f0f4adcf service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:21.597756 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1361/2678] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:01:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:01:22.730941 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a397d-2b27-4701-98fa-49225f0f6609 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:22.731549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a397d-2b27-4701-98fa-49225f0f6609 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:22.731549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a397d-2b27-4701-98fa-49225f0f6609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:22.751200 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-da8a397d-2b27-4701-98fa-49225f0f6609 None None] MFA Rules not processed for user `9245bd2079df4931846baaf7c3b54e55`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:22.752418 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-da8a397d-2b27-4701-98fa-49225f0f6609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:22.759471 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1318/2679] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:31.593754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=63 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:32.620945 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8dfab4-b4b2-43ec-82d2-1698f0f4adcf service nova] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:32.686129 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8dfab4-b4b2-43ec-82d2-1698f0f4adcf service nova] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:32.691225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48bc27-3f56-49e2-8601-dd2f0514be0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:32.691860 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48bc27-3f56-49e2-8601-dd2f0514be0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:32.692164 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48bc27-3f56-49e2-8601-dd2f0514be0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:32.692446 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48bc27-3f56-49e2-8601-dd2f0514be0b admin admin] PATH_INFO: `/v3/users/8926a23cf38d483fb31901af271f76b7` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:32.694952 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48bc27-3f56-49e2-8601-dd2f0514be0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8926a23cf38d483fb31901af271f76b7)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:32.695552 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48bc27-3f56-49e2-8601-dd2f0514be0b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:32.745474 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=57 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:32.756883 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-7b48bc27-3f56-49e2-8601-dd2f0514be0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8926a23cf38d483fb31901af271f76b7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:32.763661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-7b48bc27-3f56-49e2-8601-dd2f0514be0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8926a23cf38d483fb31901af271f76b7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:32.777096 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-7b48bc27-3f56-49e2-8601-dd2f0514be0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8926a23cf38d483fb31901af271f76b7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:32.777768 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-7b48bc27-3f56-49e2-8601-dd2f0514be0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8926a23cf38d483fb31901af271f76b7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:32.778385 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-7b48bc27-3f56-49e2-8601-dd2f0514be0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8926a23cf38d483fb31901af271f76b7'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:32.790784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1362/2680] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:01:32 2022] DELETE /identity/v3/users/8926a23cf38d483fb31901af271f76b7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:32.973837 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a397d-2b27-4701-98fa-49225f0f6609 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:32.988863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a397d-2b27-4701-98fa-49225f0f6609 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:33.049279 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33a978-4777-4871-9bb1-2d09642f6355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:33.049845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33a978-4777-4871-9bb1-2d09642f6355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:33.050128 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33a978-4777-4871-9bb1-2d09642f6355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:33.050403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc33a978-4777-4871-9bb1-2d09642f6355 admin admin] PATH_INFO: `/v3/projects/401c7126a6f24240bbcd50e86a136d88` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:33.051603 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33a978-4777-4871-9bb1-2d09642f6355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=401c7126a6f24240bbcd50e86a136d88)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:33.053492 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc33a978-4777-4871-9bb1-2d09642f6355 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:33.101670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-bc33a978-4777-4871-9bb1-2d09642f6355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '401c7126a6f24240bbcd50e86a136d88', 'request_id': 'req-bc33a978-4777-4871-9bb1-2d09642f6355'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:33.108736 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-bc33a978-4777-4871-9bb1-2d09642f6355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '401c7126a6f24240bbcd50e86a136d88', 'request_id': 'req-bc33a978-4777-4871-9bb1-2d09642f6355'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:33.115752 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-bc33a978-4777-4871-9bb1-2d09642f6355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '401c7126a6f24240bbcd50e86a136d88', 'request_id': 'req-bc33a978-4777-4871-9bb1-2d09642f6355'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:33.127093 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-bc33a978-4777-4871-9bb1-2d09642f6355 admin admin] The token cache is being invalidate because project 401c7126a6f24240bbcd50e86a136d88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:01:33.127385 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-bc33a978-4777-4871-9bb1-2d09642f6355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:33.128070 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-bc33a978-4777-4871-9bb1-2d09642f6355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:33.129938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1319/2681] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:01:32 2022] DELETE /identity/v3/projects/401c7126a6f24240bbcd50e86a136d88 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:33.134197 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48bc27-3f56-49e2-8601-dd2f0514be0b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:33.135735 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-7b48bc27-3f56-49e2-8601-dd2f0514be0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:33.149303 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48bc27-3f56-49e2-8601-dd2f0514be0b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:33.204689 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d15c08-b77a-428f-8149-efeaecc26c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:33.205325 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d15c08-b77a-428f-8149-efeaecc26c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:33.205613 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d15c08-b77a-428f-8149-efeaecc26c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:33.205889 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d15c08-b77a-428f-8149-efeaecc26c1a admin admin] PATH_INFO: `/v3/users/cb5bc84774d444f89ae3dbc5ab0f6a4f` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:33.207094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d15c08-b77a-428f-8149-efeaecc26c1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb5bc84774d444f89ae3dbc5ab0f6a4f)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:33.209031 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d15c08-b77a-428f-8149-efeaecc26c1a admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:33.234954 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:33.259347 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-36d15c08-b77a-428f-8149-efeaecc26c1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb5bc84774d444f89ae3dbc5ab0f6a4f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:33.267787 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-36d15c08-b77a-428f-8149-efeaecc26c1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb5bc84774d444f89ae3dbc5ab0f6a4f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:33.283085 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-36d15c08-b77a-428f-8149-efeaecc26c1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb5bc84774d444f89ae3dbc5ab0f6a4f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:33.283976 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-36d15c08-b77a-428f-8149-efeaecc26c1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb5bc84774d444f89ae3dbc5ab0f6a4f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:33.284584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-36d15c08-b77a-428f-8149-efeaecc26c1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb5bc84774d444f89ae3dbc5ab0f6a4f'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:33.298619 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1363/2682] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:01:33 2022] DELETE /identity/v3/users/cb5bc84774d444f89ae3dbc5ab0f6a4f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:33.476399 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33a978-4777-4871-9bb1-2d09642f6355 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:33.477921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-bc33a978-4777-4871-9bb1-2d09642f6355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:33.491070 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc33a978-4777-4871-9bb1-2d09642f6355 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:33.549397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab09df2-9b9a-40f0-9c7d-f521b05b4144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:33.550048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab09df2-9b9a-40f0-9c7d-f521b05b4144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:33.550325 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab09df2-9b9a-40f0-9c7d-f521b05b4144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:33.550628 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab09df2-9b9a-40f0-9c7d-f521b05b4144 admin admin] PATH_INFO: `/v3/projects/fe47cfe53a94443889010f625d1e4000` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:33.551771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab09df2-9b9a-40f0-9c7d-f521b05b4144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe47cfe53a94443889010f625d1e4000)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:33.553751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab09df2-9b9a-40f0-9c7d-f521b05b4144 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:33.590500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:33.603882 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8ab09df2-9b9a-40f0-9c7d-f521b05b4144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe47cfe53a94443889010f625d1e4000', 'request_id': 'req-8ab09df2-9b9a-40f0-9c7d-f521b05b4144'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:33.611708 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8ab09df2-9b9a-40f0-9c7d-f521b05b4144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe47cfe53a94443889010f625d1e4000', 'request_id': 'req-8ab09df2-9b9a-40f0-9c7d-f521b05b4144'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:33.618261 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8ab09df2-9b9a-40f0-9c7d-f521b05b4144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe47cfe53a94443889010f625d1e4000', 'request_id': 'req-8ab09df2-9b9a-40f0-9c7d-f521b05b4144'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:33.628986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8ab09df2-9b9a-40f0-9c7d-f521b05b4144 admin admin] The token cache is being invalidate because project fe47cfe53a94443889010f625d1e4000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:01:33.629270 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8ab09df2-9b9a-40f0-9c7d-f521b05b4144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:33.629896 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8ab09df2-9b9a-40f0-9c7d-f521b05b4144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:01:33.631426 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1320/2683] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:01:33 2022] DELETE /identity/v3/projects/fe47cfe53a94443889010f625d1e4000 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:33.636225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f039358-e7c2-479c-9458-2a25c10da1bd None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:33.636701 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f039358-e7c2-479c-9458-2a25c10da1bd None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:33.636800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f039358-e7c2-479c-9458-2a25c10da1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:33.656479 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-3f039358-e7c2-479c-9458-2a25c10da1bd None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:33.657673 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-3f039358-e7c2-479c-9458-2a25c10da1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:33.729342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1364/2684] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:33.737820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75760229-cf99-49d3-83b3-46f70f4f765c None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:33.738149 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75760229-cf99-49d3-83b3-46f70f4f765c None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:33.738427 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75760229-cf99-49d3-83b3-46f70f4f765c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:33.758983 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-75760229-cf99-49d3-83b3-46f70f4f765c None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:33.760180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-75760229-cf99-49d3-83b3-46f70f4f765c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:33.768468 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1321/2685] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:01:33.772492 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f039358-e7c2-479c-9458-2a25c10da1bd None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:33.782214 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f039358-e7c2-479c-9458-2a25c10da1bd None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:33.786979 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cab9a-be54-4e38-8f6b-b62e8248fde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:33.787598 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cab9a-be54-4e38-8f6b-b62e8248fde0 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:33.787880 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cab9a-be54-4e38-8f6b-b62e8248fde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:33.788154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cab9a-be54-4e38-8f6b-b62e8248fde0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:33.788969 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cab9a-be54-4e38-8f6b-b62e8248fde0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 13:01:33.789283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cab9a-be54-4e38-8f6b-b62e8248fde0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:33.790900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cab9a-be54-4e38-8f6b-b62e8248fde0 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:33.803706 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1365/2686] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 13:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 13:01:33.807824 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75760229-cf99-49d3-83b3-46f70f4f765c None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:33.818407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75760229-cf99-49d3-83b3-46f70f4f765c None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:33.824044 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09b998-01d7-45b2-ad39-2d739ec9a1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:33.824769 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09b998-01d7-45b2-ad39-2d739ec9a1bc admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:33.825118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09b998-01d7-45b2-ad39-2d739ec9a1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:33.825457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09b998-01d7-45b2-ad39-2d739ec9a1bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:33.826200 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09b998-01d7-45b2-ad39-2d739ec9a1bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:33.828255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09b998-01d7-45b2-ad39-2d739ec9a1bc admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:33.855021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1322/2687] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:01:33 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:01:33.859720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cab9a-be54-4e38-8f6b-b62e8248fde0 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:33.869698 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cab9a-be54-4e38-8f6b-b62e8248fde0 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:33.925440 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00c339-07d8-49a0-8f52-3f74741358ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:33.926092 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00c339-07d8-49a0-8f52-3f74741358ac admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:33.926374 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00c339-07d8-49a0-8f52-3f74741358ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:33.926689 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00c339-07d8-49a0-8f52-3f74741358ac admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:33.927407 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00c339-07d8-49a0-8f52-3f74741358ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:33.929315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00c339-07d8-49a0-8f52-3f74741358ac admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:33.951201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1366/2688] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:01:33 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 13:01:33.955681 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09b998-01d7-45b2-ad39-2d739ec9a1bc admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:33.966056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09b998-01d7-45b2-ad39-2d739ec9a1bc admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:33.971296 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d0f03-769c-4d6e-b98a-47b75951ba9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:33.971898 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d0f03-769c-4d6e-b98a-47b75951ba9d admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:33.972178 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d0f03-769c-4d6e-b98a-47b75951ba9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:33.972450 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d0f03-769c-4d6e-b98a-47b75951ba9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:33.973192 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d0f03-769c-4d6e-b98a-47b75951ba9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:33.974805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d0f03-769c-4d6e-b98a-47b75951ba9d admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:33.984542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1323/2689] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:01:33.988243 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00c339-07d8-49a0-8f52-3f74741358ac admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:33.998139 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00c339-07d8-49a0-8f52-3f74741358ac admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:34.002713 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93d4d5-1f2d-42ed-beee-5b0e5c6e83d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:34.003311 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93d4d5-1f2d-42ed-beee-5b0e5c6e83d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:34.003586 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93d4d5-1f2d-42ed-beee-5b0e5c6e83d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:34.003853 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93d4d5-1f2d-42ed-beee-5b0e5c6e83d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:34.004846 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93d4d5-1f2d-42ed-beee-5b0e5c6e83d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:34.006428 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93d4d5-1f2d-42ed-beee-5b0e5c6e83d6 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:34.017319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1367/2690] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:01:34.021489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d0f03-769c-4d6e-b98a-47b75951ba9d admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:34.031900 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d0f03-769c-4d6e-b98a-47b75951ba9d admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:34.036485 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425bcb6-6b61-4b9e-84f1-b76ac7973a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:34.037135 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425bcb6-6b61-4b9e-84f1-b76ac7973a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:34.037416 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425bcb6-6b61-4b9e-84f1-b76ac7973a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:34.037692 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425bcb6-6b61-4b9e-84f1-b76ac7973a59 admin admin] PATH_INFO: `/v3/projects/bfd5db2ef628483f8abdb065b8bb0d6d/users/f621dfa05b5243e7b3601e4701f948ab/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:34.052071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425bcb6-6b61-4b9e-84f1-b76ac7973a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd5db2ef628483f8abdb065b8bb0d6d, user_id=f621dfa05b5243e7b3601e4701f948ab, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:34.054694 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425bcb6-6b61-4b9e-84f1-b76ac7973a59 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:34.067548 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1324/2691] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:01:34 2022] PUT /identity/v3/projects/bfd5db2ef628483f8abdb065b8bb0d6d/users/f621dfa05b5243e7b3601e4701f948ab/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:01:34.080075 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70606518-4a94-4205-8a58-7c5148c0897a None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:34.080360 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70606518-4a94-4205-8a58-7c5148c0897a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:34.080631 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70606518-4a94-4205-8a58-7c5148c0897a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:34.088661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:34.110351 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-70606518-4a94-4205-8a58-7c5148c0897a None None] MFA Rules not processed for user `f621dfa05b5243e7b3601e4701f948ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:01:34.111612 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-70606518-4a94-4205-8a58-7c5148c0897a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:34.174538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1368/2692] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:01:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 13:01:34.185998 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425bcb6-6b61-4b9e-84f1-b76ac7973a59 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:34.188242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f425bcb6-6b61-4b9e-84f1-b76ac7973a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:34.267551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425bcb6-6b61-4b9e-84f1-b76ac7973a59 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:34.271577 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e970ab-d988-45f3-9488-dc94006584ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:34.272185 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e970ab-d988-45f3-9488-dc94006584ed service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:34.272464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e970ab-d988-45f3-9488-dc94006584ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:34.272736 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e970ab-d988-45f3-9488-dc94006584ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:34.281612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e970ab-d988-45f3-9488-dc94006584ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:34.283050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e970ab-d988-45f3-9488-dc94006584ed service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:34.291771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1325/2693] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:01:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:01:41.406079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70606518-4a94-4205-8a58-7c5148c0897a None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:41.407529 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-70606518-4a94-4205-8a58-7c5148c0897a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:41.496679 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70606518-4a94-4205-8a58-7c5148c0897a None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:41.502064 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f6918-a41b-4d60-8dbc-b8400c8319dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5094263ac471481db48783b19192e01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:41.502718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f6918-a41b-4d60-8dbc-b8400c8319dc service placement] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:41.503000 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f6918-a41b-4d60-8dbc-b8400c8319dc service placement] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:41.503271 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f6918-a41b-4d60-8dbc-b8400c8319dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:41.505046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-183f6918-a41b-4d60-8dbc-b8400c8319dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:41.516771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f6918-a41b-4d60-8dbc-b8400c8319dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:41.518322 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f6918-a41b-4d60-8dbc-b8400c8319dc service placement] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:41.528376 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1369/2694] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:01:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:01:44.287703 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=74,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:51.524102 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=71 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:01:59.682859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e970ab-d988-45f3-9488-dc94006584ed service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:01:59.684301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-24e970ab-d988-45f3-9488-dc94006584ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:59.759516 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e970ab-d988-45f3-9488-dc94006584ed service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:01:59.763302 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3075bd8-32f2-412f-92dc-bd4d5f4ddefd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:01:59.763915 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3075bd8-32f2-412f-92dc-bd4d5f4ddefd service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:01:59.764195 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3075bd8-32f2-412f-92dc-bd4d5f4ddefd service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:01:59.764470 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3075bd8-32f2-412f-92dc-bd4d5f4ddefd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:01:59.766217 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d3075bd8-32f2-412f-92dc-bd4d5f4ddefd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:01:59.826768 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3075bd8-32f2-412f-92dc-bd4d5f4ddefd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:01:59.828184 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3075bd8-32f2-412f-92dc-bd4d5f4ddefd service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:01:59.837678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1326/2695] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:01:59 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:02:09.527533 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f6918-a41b-4d60-8dbc-b8400c8319dc service placement] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:09.602724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f6918-a41b-4d60-8dbc-b8400c8319dc service placement] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:09.607228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb86c8-6e77-4b2b-b270-86e0a1bf7fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:09.607868 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb86c8-6e77-4b2b-b270-86e0a1bf7fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:09.608691 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb86c8-6e77-4b2b-b270-86e0a1bf7fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:09.608971 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb86c8-6e77-4b2b-b270-86e0a1bf7fc8 admin admin] PATH_INFO: `/v3/users/f621dfa05b5243e7b3601e4701f948ab` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:09.610269 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb86c8-6e77-4b2b-b270-86e0a1bf7fc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f621dfa05b5243e7b3601e4701f948ab)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:09.612228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb86c8-6e77-4b2b-b270-86e0a1bf7fc8 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:09.664352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-41cb86c8-6e77-4b2b-b270-86e0a1bf7fc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f621dfa05b5243e7b3601e4701f948ab'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:09.672038 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-41cb86c8-6e77-4b2b-b270-86e0a1bf7fc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f621dfa05b5243e7b3601e4701f948ab'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:09.684630 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-41cb86c8-6e77-4b2b-b270-86e0a1bf7fc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f621dfa05b5243e7b3601e4701f948ab'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:09.685373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-41cb86c8-6e77-4b2b-b270-86e0a1bf7fc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f621dfa05b5243e7b3601e4701f948ab'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:09.685989 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-41cb86c8-6e77-4b2b-b270-86e0a1bf7fc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f621dfa05b5243e7b3601e4701f948ab'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:09.700725 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1370/2696] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:02:09 2022] DELETE /identity/v3/users/f621dfa05b5243e7b3601e4701f948ab => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:02:09.705234 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f7a39-f3d8-49f3-9714-15a19b5d69d5 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:09.705759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f7a39-f3d8-49f3-9714-15a19b5d69d5 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:09.705759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f7a39-f3d8-49f3-9714-15a19b5d69d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:09.724180 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-a32f7a39-f3d8-49f3-9714-15a19b5d69d5 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:02:09.725626 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a32f7a39-f3d8-49f3-9714-15a19b5d69d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:02:09.785058 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1327/2697] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:02:09.795440 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb86c8-6e77-4b2b-b270-86e0a1bf7fc8 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:09.797508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-41cb86c8-6e77-4b2b-b270-86e0a1bf7fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:02:09.860622 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb86c8-6e77-4b2b-b270-86e0a1bf7fc8 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:09.864288 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c2186-86a5-48b6-9b6f-f82ebb1f000d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:09.864925 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c2186-86a5-48b6-9b6f-f82ebb1f000d service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:09.865204 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c2186-86a5-48b6-9b6f-f82ebb1f000d service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:09.865476 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c2186-86a5-48b6-9b6f-f82ebb1f000d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:09.876634 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c2186-86a5-48b6-9b6f-f82ebb1f000d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:09.878545 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c2186-86a5-48b6-9b6f-f82ebb1f000d service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:09.890010 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1371/2698] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:02:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:02:10.244350 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f7a39-f3d8-49f3-9714-15a19b5d69d5 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:10.246115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a32f7a39-f3d8-49f3-9714-15a19b5d69d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:02:10.265287 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f7a39-f3d8-49f3-9714-15a19b5d69d5 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:10.269848 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a8608-1145-4bc1-8445-dcf37561d1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:10.270531 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a8608-1145-4bc1-8445-dcf37561d1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:10.270798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a8608-1145-4bc1-8445-dcf37561d1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:10.271074 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a8608-1145-4bc1-8445-dcf37561d1ad admin admin] PATH_INFO: `/v3/projects/bfd5db2ef628483f8abdb065b8bb0d6d` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:10.272228 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a8608-1145-4bc1-8445-dcf37561d1ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfd5db2ef628483f8abdb065b8bb0d6d)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:10.274109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a8608-1145-4bc1-8445-dcf37561d1ad admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:10.314811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:02:10.333085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-909a8608-1145-4bc1-8445-dcf37561d1ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfd5db2ef628483f8abdb065b8bb0d6d', 'request_id': 'req-909a8608-1145-4bc1-8445-dcf37561d1ad'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:10.340303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-909a8608-1145-4bc1-8445-dcf37561d1ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfd5db2ef628483f8abdb065b8bb0d6d', 'request_id': 'req-909a8608-1145-4bc1-8445-dcf37561d1ad'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:10.347467 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-909a8608-1145-4bc1-8445-dcf37561d1ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfd5db2ef628483f8abdb065b8bb0d6d', 'request_id': 'req-909a8608-1145-4bc1-8445-dcf37561d1ad'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:10.359283 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-909a8608-1145-4bc1-8445-dcf37561d1ad admin admin] The token cache is being invalidate because project bfd5db2ef628483f8abdb065b8bb0d6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:02:10.359793 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-909a8608-1145-4bc1-8445-dcf37561d1ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:10.360538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-909a8608-1145-4bc1-8445-dcf37561d1ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:10.362086 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1328/2699] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:02:10 2022] DELETE /identity/v3/projects/bfd5db2ef628483f8abdb065b8bb0d6d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:02:10.367116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc1467-936d-4033-a63e-16cb8719de32 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:10.367413 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc1467-936d-4033-a63e-16cb8719de32 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:10.367679 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc1467-936d-4033-a63e-16cb8719de32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:10.388073 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-ebbc1467-936d-4033-a63e-16cb8719de32 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:02:10.389872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ebbc1467-936d-4033-a63e-16cb8719de32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:02:10.407880 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:02:10.455747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1372/2700] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:02:10.463846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8282457-0bb7-4e12-9f41-4c6315cd6d5d None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:10.464131 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8282457-0bb7-4e12-9f41-4c6315cd6d5d None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:10.464404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8282457-0bb7-4e12-9f41-4c6315cd6d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:10.485036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-e8282457-0bb7-4e12-9f41-4c6315cd6d5d None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:02:10.486313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e8282457-0bb7-4e12-9f41-4c6315cd6d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:02:10.494111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1329/2701] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:02:10.498345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc1467-936d-4033-a63e-16cb8719de32 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:10.508899 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc1467-936d-4033-a63e-16cb8719de32 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:10.513464 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e66ad0-83a0-4522-8825-6315ea7603d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:10.514058 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e66ad0-83a0-4522-8825-6315ea7603d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:10.514961 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e66ad0-83a0-4522-8825-6315ea7603d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:10.515233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e66ad0-83a0-4522-8825-6315ea7603d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:10.515929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e66ad0-83a0-4522-8825-6315ea7603d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 13:02:10.516229 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e66ad0-83a0-4522-8825-6315ea7603d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:10.517802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e66ad0-83a0-4522-8825-6315ea7603d0 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:10.529349 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1373/2702] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 13:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 13:02:10.534270 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8282457-0bb7-4e12-9f41-4c6315cd6d5d None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:10.543589 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8282457-0bb7-4e12-9f41-4c6315cd6d5d None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:10.548652 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccc565-eb73-4649-9c3b-b356e6f1ac2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:10.549261 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccc565-eb73-4649-9c3b-b356e6f1ac2f admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:10.549541 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccc565-eb73-4649-9c3b-b356e6f1ac2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:10.549811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccc565-eb73-4649-9c3b-b356e6f1ac2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:10.550530 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ccc565-eb73-4649-9c3b-b356e6f1ac2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:10.552461 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ccc565-eb73-4649-9c3b-b356e6f1ac2f admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:10.580565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1330/2703] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:02:10 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:02:10.587224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e66ad0-83a0-4522-8825-6315ea7603d0 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:10.598156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e66ad0-83a0-4522-8825-6315ea7603d0 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:10.662071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f2a56-37d2-43d9-9d5a-8c7735acd82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:10.662847 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f2a56-37d2-43d9-9d5a-8c7735acd82e admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:10.663432 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f2a56-37d2-43d9-9d5a-8c7735acd82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:10.663811 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f2a56-37d2-43d9-9d5a-8c7735acd82e admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:10.664590 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f2a56-37d2-43d9-9d5a-8c7735acd82e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:10.666981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f2a56-37d2-43d9-9d5a-8c7735acd82e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:10.698062 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1374/2704] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:02:10 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:02:10.703281 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccc565-eb73-4649-9c3b-b356e6f1ac2f admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:10.713917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccc565-eb73-4649-9c3b-b356e6f1ac2f admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:10.721791 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28bbc0-f67f-4600-9a48-1a6ad47088de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:10.722740 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28bbc0-f67f-4600-9a48-1a6ad47088de admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:10.723215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28bbc0-f67f-4600-9a48-1a6ad47088de admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:10.723686 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28bbc0-f67f-4600-9a48-1a6ad47088de admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:10.724569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28bbc0-f67f-4600-9a48-1a6ad47088de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:10.726344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28bbc0-f67f-4600-9a48-1a6ad47088de admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:10.738572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1331/2705] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:02:10.742499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f2a56-37d2-43d9-9d5a-8c7735acd82e admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:10.752797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f2a56-37d2-43d9-9d5a-8c7735acd82e admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:10.758076 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1770aa-a2bf-48d0-8fc0-a814e7f53d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:10.758720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1770aa-a2bf-48d0-8fc0-a814e7f53d65 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:10.759008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1770aa-a2bf-48d0-8fc0-a814e7f53d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:10.759279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1770aa-a2bf-48d0-8fc0-a814e7f53d65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:10.759975 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1770aa-a2bf-48d0-8fc0-a814e7f53d65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:10.761563 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1770aa-a2bf-48d0-8fc0-a814e7f53d65 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:10.773123 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1375/2706] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:02:10.777182 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28bbc0-f67f-4600-9a48-1a6ad47088de admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:10.787692 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28bbc0-f67f-4600-9a48-1a6ad47088de admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:10.792372 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e40b1-6e5d-4062-aad1-5d06f14fbe94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:10.793172 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e40b1-6e5d-4062-aad1-5d06f14fbe94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:10.793517 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e40b1-6e5d-4062-aad1-5d06f14fbe94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:10.793859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e40b1-6e5d-4062-aad1-5d06f14fbe94 admin admin] PATH_INFO: `/v3/projects/11d95ca2887a41a7afcaeee22a6b5a4a/users/a3c1ea0e5c124cd89ba512e8f7837576/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:10.810875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e40b1-6e5d-4062-aad1-5d06f14fbe94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11d95ca2887a41a7afcaeee22a6b5a4a, user_id=a3c1ea0e5c124cd89ba512e8f7837576, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:10.814075 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e40b1-6e5d-4062-aad1-5d06f14fbe94 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:10.828950 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1332/2707] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:02:10 2022] PUT /identity/v3/projects/11d95ca2887a41a7afcaeee22a6b5a4a/users/a3c1ea0e5c124cd89ba512e8f7837576/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:02:10.840122 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee374f15-59b0-47d1-a6d9-53216f2a97d1 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:10.840391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee374f15-59b0-47d1-a6d9-53216f2a97d1 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:10.840703 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee374f15-59b0-47d1-a6d9-53216f2a97d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:10.871253 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-ee374f15-59b0-47d1-a6d9-53216f2a97d1 None None] MFA Rules not processed for user `a3c1ea0e5c124cd89ba512e8f7837576`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:02:10.872501 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-ee374f15-59b0-47d1-a6d9-53216f2a97d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:02:10.964354 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1376/2708] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:02:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:02:10.976566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e40b1-6e5d-4062-aad1-5d06f14fbe94 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:10.978407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-375e40b1-6e5d-4062-aad1-5d06f14fbe94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:02:11.048446 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e40b1-6e5d-4062-aad1-5d06f14fbe94 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:11.052207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00ac88-64a6-4edd-96c6-d94616313707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:11.052853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00ac88-64a6-4edd-96c6-d94616313707 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:11.053145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00ac88-64a6-4edd-96c6-d94616313707 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:11.053397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00ac88-64a6-4edd-96c6-d94616313707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:11.063967 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00ac88-64a6-4edd-96c6-d94616313707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:11.065406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00ac88-64a6-4edd-96c6-d94616313707 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:11.075904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1333/2709] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:02:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:02:13.901942 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab605a7-f881-4d58-bddd-00e7d46cc112 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:13.902383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab605a7-f881-4d58-bddd-00e7d46cc112 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:13.902557 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab605a7-f881-4d58-bddd-00e7d46cc112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:13.922873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-3ab605a7-f881-4d58-bddd-00e7d46cc112 None None] MFA Rules not processed for user `fd489ef668494ec98cf9d167a1ded3ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:02:13.924126 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-3ab605a7-f881-4d58-bddd-00e7d46cc112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:02:13.937210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:02:13.984605 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1377/2710] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:02:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:02:21.034272 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00ac88-64a6-4edd-96c6-d94616313707 service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:21.085041 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:02:21.097644 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00ac88-64a6-4edd-96c6-d94616313707 service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:21.102566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c64da-2f82-459b-aab5-76f85cf66e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:21.103191 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c64da-2f82-459b-aab5-76f85cf66e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:21.103465 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c64da-2f82-459b-aab5-76f85cf66e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:21.103733 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c64da-2f82-459b-aab5-76f85cf66e8e admin admin] PATH_INFO: `/v3/users/a3c1ea0e5c124cd89ba512e8f7837576` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:21.104905 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c64da-2f82-459b-aab5-76f85cf66e8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3c1ea0e5c124cd89ba512e8f7837576)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:21.106811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c64da-2f82-459b-aab5-76f85cf66e8e admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:21.151167 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-049c64da-2f82-459b-aab5-76f85cf66e8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3c1ea0e5c124cd89ba512e8f7837576'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:21.162462 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-049c64da-2f82-459b-aab5-76f85cf66e8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3c1ea0e5c124cd89ba512e8f7837576'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:21.169219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-049c64da-2f82-459b-aab5-76f85cf66e8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3c1ea0e5c124cd89ba512e8f7837576'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:21.169922 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-049c64da-2f82-459b-aab5-76f85cf66e8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3c1ea0e5c124cd89ba512e8f7837576'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:21.177631 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-049c64da-2f82-459b-aab5-76f85cf66e8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3c1ea0e5c124cd89ba512e8f7837576'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:21.183466 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1334/2711] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:02:21 2022] DELETE /identity/v3/users/a3c1ea0e5c124cd89ba512e8f7837576 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:02:21.188610 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da512d44-3040-4278-9d1c-b498e0e7edff None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:21.189133 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da512d44-3040-4278-9d1c-b498e0e7edff None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:21.189247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da512d44-3040-4278-9d1c-b498e0e7edff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:21.207534 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-da512d44-3040-4278-9d1c-b498e0e7edff None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:02:21.208720 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-da512d44-3040-4278-9d1c-b498e0e7edff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:02:21.268685 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1378/2712] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:02:21.278697 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c64da-2f82-459b-aab5-76f85cf66e8e admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:21.280612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-049c64da-2f82-459b-aab5-76f85cf66e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:02:21.337844 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c64da-2f82-459b-aab5-76f85cf66e8e admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:21.341427 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde7fe2-3821-442d-b572-3d643aa3fd78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:21.342034 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde7fe2-3821-442d-b572-3d643aa3fd78 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:21.342832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde7fe2-3821-442d-b572-3d643aa3fd78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:21.343119 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde7fe2-3821-442d-b572-3d643aa3fd78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:21.351931 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde7fe2-3821-442d-b572-3d643aa3fd78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:21.353332 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cde7fe2-3821-442d-b572-3d643aa3fd78 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:21.363350 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1335/2713] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:02:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:02:21.635445 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da512d44-3040-4278-9d1c-b498e0e7edff None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:21.636868 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-da512d44-3040-4278-9d1c-b498e0e7edff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:02:21.650353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da512d44-3040-4278-9d1c-b498e0e7edff None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:21.674499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:21.674499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:21.674499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:21.674499 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc admin admin] PATH_INFO: `/v3/projects/11d95ca2887a41a7afcaeee22a6b5a4a` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:21.674902 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc admin admin] RBAC: Authorizing `identity:delete_project(project_id=11d95ca2887a41a7afcaeee22a6b5a4a)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:21.674902 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:21.719776 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11d95ca2887a41a7afcaeee22a6b5a4a', 'request_id': 'req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:21.726682 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11d95ca2887a41a7afcaeee22a6b5a4a', 'request_id': 'req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:21.732640 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11d95ca2887a41a7afcaeee22a6b5a4a', 'request_id': 'req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:21.742659 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc admin admin] The token cache is being invalidate because project 11d95ca2887a41a7afcaeee22a6b5a4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:02:21.742939 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:21.743554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:21.745041 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1379/2714] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:02:21 2022] DELETE /identity/v3/projects/11d95ca2887a41a7afcaeee22a6b5a4a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:02:31.358452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:02:31.739856 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:02:51.796572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde7fe2-3821-442d-b572-3d643aa3fd78 service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:51.798037 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1cde7fe2-3821-442d-b572-3d643aa3fd78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:02:51.809380 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde7fe2-3821-442d-b572-3d643aa3fd78 service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:51.856973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5216d-42c4-4ec6-87ea-44031b1a0008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:51.857583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5216d-42c4-4ec6-87ea-44031b1a0008 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:51.857859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5216d-42c4-4ec6-87ea-44031b1a0008 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:51.858127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5216d-42c4-4ec6-87ea-44031b1a0008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:51.859938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ebf5216d-42c4-4ec6-87ea-44031b1a0008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:02:51.922052 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5216d-42c4-4ec6-87ea-44031b1a0008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:51.923515 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5216d-42c4-4ec6-87ea-44031b1a0008 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:51.933406 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1336/2715] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:02:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:02:58.080337 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:58.081796 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:02:58.096443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bbe02-0ba9-4ec9-92e7-95ec9f7e8ffc admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:58.101179 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817e885-6cc7-49c4-93c2-ada1074032ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:58.101821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817e885-6cc7-49c4-93c2-ada1074032ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:58.102099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817e885-6cc7-49c4-93c2-ada1074032ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:58.102369 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8817e885-6cc7-49c4-93c2-ada1074032ae admin admin] PATH_INFO: `/v3/users/8ccd455b417549e3b00439cee4577d11` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:58.103621 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817e885-6cc7-49c4-93c2-ada1074032ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ccd455b417549e3b00439cee4577d11)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:58.105526 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8817e885-6cc7-49c4-93c2-ada1074032ae admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:58.150523 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-8817e885-6cc7-49c4-93c2-ada1074032ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ccd455b417549e3b00439cee4577d11'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.157812 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-8817e885-6cc7-49c4-93c2-ada1074032ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ccd455b417549e3b00439cee4577d11'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.169523 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-8817e885-6cc7-49c4-93c2-ada1074032ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ccd455b417549e3b00439cee4577d11'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.170178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-8817e885-6cc7-49c4-93c2-ada1074032ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ccd455b417549e3b00439cee4577d11'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.170792 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-8817e885-6cc7-49c4-93c2-ada1074032ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ccd455b417549e3b00439cee4577d11'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.183623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1380/2716] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:02:58 2022] DELETE /identity/v3/users/8ccd455b417549e3b00439cee4577d11 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:02:58.355241 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5216d-42c4-4ec6-87ea-44031b1a0008 service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:58.366324 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5216d-42c4-4ec6-87ea-44031b1a0008 service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:58.422923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6d27b-dcd5-411e-b762-c76660f90644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:58.423560 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6d27b-dcd5-411e-b762-c76660f90644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:58.423840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6d27b-dcd5-411e-b762-c76660f90644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:58.424115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d6d27b-dcd5-411e-b762-c76660f90644 admin admin] PATH_INFO: `/v3/projects/9c17656e9e664e898288ec163f11fc63` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:58.425554 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6d27b-dcd5-411e-b762-c76660f90644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c17656e9e664e898288ec163f11fc63)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:58.427483 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d6d27b-dcd5-411e-b762-c76660f90644 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:58.472509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-69d6d27b-dcd5-411e-b762-c76660f90644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c17656e9e664e898288ec163f11fc63', 'request_id': 'req-69d6d27b-dcd5-411e-b762-c76660f90644'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.474944 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:02:58.479766 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-69d6d27b-dcd5-411e-b762-c76660f90644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c17656e9e664e898288ec163f11fc63', 'request_id': 'req-69d6d27b-dcd5-411e-b762-c76660f90644'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.486655 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-69d6d27b-dcd5-411e-b762-c76660f90644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c17656e9e664e898288ec163f11fc63', 'request_id': 'req-69d6d27b-dcd5-411e-b762-c76660f90644'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.496654 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-69d6d27b-dcd5-411e-b762-c76660f90644 admin admin] The token cache is being invalidate because project 9c17656e9e664e898288ec163f11fc63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:02:58.496940 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-69d6d27b-dcd5-411e-b762-c76660f90644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.497562 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-69d6d27b-dcd5-411e-b762-c76660f90644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.499080 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1337/2717] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:02:58 2022] DELETE /identity/v3/projects/9c17656e9e664e898288ec163f11fc63 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:02:58.503297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817e885-6cc7-49c4-93c2-ada1074032ae admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:58.504693 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8817e885-6cc7-49c4-93c2-ada1074032ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:02:58.518564 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8817e885-6cc7-49c4-93c2-ada1074032ae admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:58.573190 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec7eb6-ce96-4fe6-a7d1-b8c2411f7841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:58.573819 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec7eb6-ce96-4fe6-a7d1-b8c2411f7841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:58.574100 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec7eb6-ce96-4fe6-a7d1-b8c2411f7841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:58.574374 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec7eb6-ce96-4fe6-a7d1-b8c2411f7841 admin admin] PATH_INFO: `/v3/users/b142cc45cfab477eb8dfb40f817c9f27` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:58.575584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec7eb6-ce96-4fe6-a7d1-b8c2411f7841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b142cc45cfab477eb8dfb40f817c9f27)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:58.577477 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec7eb6-ce96-4fe6-a7d1-b8c2411f7841 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:58.624161 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e1ec7eb6-ce96-4fe6-a7d1-b8c2411f7841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b142cc45cfab477eb8dfb40f817c9f27'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.631967 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e1ec7eb6-ce96-4fe6-a7d1-b8c2411f7841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b142cc45cfab477eb8dfb40f817c9f27'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.644278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e1ec7eb6-ce96-4fe6-a7d1-b8c2411f7841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b142cc45cfab477eb8dfb40f817c9f27'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.644928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e1ec7eb6-ce96-4fe6-a7d1-b8c2411f7841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b142cc45cfab477eb8dfb40f817c9f27'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.645566 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-e1ec7eb6-ce96-4fe6-a7d1-b8c2411f7841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b142cc45cfab477eb8dfb40f817c9f27'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.647251 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:02:58.660441 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1381/2718] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:02:58 2022] DELETE /identity/v3/users/b142cc45cfab477eb8dfb40f817c9f27 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:02:58.834301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6d27b-dcd5-411e-b762-c76660f90644 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:02:58.835777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-69d6d27b-dcd5-411e-b762-c76660f90644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:02:58.849031 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d6d27b-dcd5-411e-b762-c76660f90644 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:02:58.908320 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4682d9a-0f60-4faa-ac72-62771f6a93d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:02:58.908958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4682d9a-0f60-4faa-ac72-62771f6a93d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:02:58.909242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4682d9a-0f60-4faa-ac72-62771f6a93d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:02:58.909513 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4682d9a-0f60-4faa-ac72-62771f6a93d0 admin admin] PATH_INFO: `/v3/projects/05a3a60014f445feb168c25b4109cc9b` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:02:58.910694 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4682d9a-0f60-4faa-ac72-62771f6a93d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05a3a60014f445feb168c25b4109cc9b)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:02:58.912569 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4682d9a-0f60-4faa-ac72-62771f6a93d0 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:02:58.961182 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f4682d9a-0f60-4faa-ac72-62771f6a93d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05a3a60014f445feb168c25b4109cc9b', 'request_id': 'req-f4682d9a-0f60-4faa-ac72-62771f6a93d0'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.970713 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f4682d9a-0f60-4faa-ac72-62771f6a93d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05a3a60014f445feb168c25b4109cc9b', 'request_id': 'req-f4682d9a-0f60-4faa-ac72-62771f6a93d0'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.978214 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f4682d9a-0f60-4faa-ac72-62771f6a93d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05a3a60014f445feb168c25b4109cc9b', 'request_id': 'req-f4682d9a-0f60-4faa-ac72-62771f6a93d0'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.990688 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f4682d9a-0f60-4faa-ac72-62771f6a93d0 admin admin] The token cache is being invalidate because project 05a3a60014f445feb168c25b4109cc9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:02:58.991023 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f4682d9a-0f60-4faa-ac72-62771f6a93d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.991707 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-f4682d9a-0f60-4faa-ac72-62771f6a93d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:02:58.993261 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1338/2719] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:02:58 2022] DELETE /identity/v3/projects/05a3a60014f445feb168c25b4109cc9b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:03:03.418714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec7eb6-ce96-4fe6-a7d1-b8c2411f7841 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:03:03.420122 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e1ec7eb6-ce96-4fe6-a7d1-b8c2411f7841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:03:03.476581 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec7eb6-ce96-4fe6-a7d1-b8c2411f7841 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:03:03.524649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e874ab-cb83-4629-b886-ef44451b8783 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:03:03.525261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e874ab-cb83-4629-b886-ef44451b8783 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:03:03.525542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e874ab-cb83-4629-b886-ef44451b8783 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:03:03.525817 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e874ab-cb83-4629-b886-ef44451b8783 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:03:03.527623 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-63e874ab-cb83-4629-b886-ef44451b8783 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:03:03.597334 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e874ab-cb83-4629-b886-ef44451b8783 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:03:03.598703 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e874ab-cb83-4629-b886-ef44451b8783 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:03:03.609630 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1382/2720] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:03:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:03:08.987474 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=10,keystone:SELECT=57,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:03:11.394215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4682d9a-0f60-4faa-ac72-62771f6a93d0 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:03:11.395740 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f4682d9a-0f60-4faa-ac72-62771f6a93d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:03:11.408266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4682d9a-0f60-4faa-ac72-62771f6a93d0 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:03:11.411795 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b9448-46b3-4c17-b0c3-998e1d16a7fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:03:11.412393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b9448-46b3-4c17-b0c3-998e1d16a7fb service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:03:11.412672 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b9448-46b3-4c17-b0c3-998e1d16a7fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:03:11.412941 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b9448-46b3-4c17-b0c3-998e1d16a7fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:03:11.414653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a63b9448-46b3-4c17-b0c3-998e1d16a7fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:03:11.428465 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b9448-46b3-4c17-b0c3-998e1d16a7fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:03:11.429863 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63b9448-46b3-4c17-b0c3-998e1d16a7fb service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:03:11.441056 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1339/2721] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:03:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:03:13.604722 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=76 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:03:18.267754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa56cc-ccde-4824-a4f5-103ae6c2b756 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:03:18.268372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa56cc-ccde-4824-a4f5-103ae6c2b756 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:03:18.268372 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa56cc-ccde-4824-a4f5-103ae6c2b756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:03:18.287167 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-2bfa56cc-ccde-4824-a4f5-103ae6c2b756 None None] MFA Rules not processed for user `fd489ef668494ec98cf9d167a1ded3ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:03:18.288355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-2bfa56cc-ccde-4824-a4f5-103ae6c2b756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:03:18.341052 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1383/2722] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:03:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:03:18.956521 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b9448-46b3-4c17-b0c3-998e1d16a7fb service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:03:18.958002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a63b9448-46b3-4c17-b0c3-998e1d16a7fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:03:19.029477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b9448-46b3-4c17-b0c3-998e1d16a7fb service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:03:19.034463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6acd2-8131-4a97-ba8d-a66d3d251361 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5094263ac471481db48783b19192e01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:03:19.035104 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6acd2-8131-4a97-ba8d-a66d3d251361 service placement] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:03:19.035384 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6acd2-8131-4a97-ba8d-a66d3d251361 service placement] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:03:19.035653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6acd2-8131-4a97-ba8d-a66d3d251361 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:03:19.037404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-fab6acd2-8131-4a97-ba8d-a66d3d251361 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:03:19.048838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6acd2-8131-4a97-ba8d-a66d3d251361 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:03:19.050391 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6acd2-8131-4a97-ba8d-a66d3d251361 service placement] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:03:19.060087 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1340/2723] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:03:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:03:28.334087 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=25 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:03:29.055972 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=38 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:04:06.480588 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa56cc-ccde-4824-a4f5-103ae6c2b756 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:06.482254 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-2bfa56cc-ccde-4824-a4f5-103ae6c2b756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:04:06.538812 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa56cc-ccde-4824-a4f5-103ae6c2b756 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:06.542570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5424c7d-c172-4f69-92c1-79ec2ca104f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:06.543180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5424c7d-c172-4f69-92c1-79ec2ca104f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:06.543461 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5424c7d-c172-4f69-92c1-79ec2ca104f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:06.543732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5424c7d-c172-4f69-92c1-79ec2ca104f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:06.545454 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-e5424c7d-c172-4f69-92c1-79ec2ca104f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:04:06.554846 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5424c7d-c172-4f69-92c1-79ec2ca104f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:06.556247 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5424c7d-c172-4f69-92c1-79ec2ca104f2 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:06.564094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1384/2724] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:04:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:04:15.059264 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6acd2-8131-4a97-ba8d-a66d3d251361 service placement] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:15.061512 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-fab6acd2-8131-4a97-ba8d-a66d3d251361 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:04:15.080942 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6acd2-8131-4a97-ba8d-a66d3d251361 service placement] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:15.089064 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b571685-b695-4a36-aeeb-50df29cc2354 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:15.090005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b571685-b695-4a36-aeeb-50df29cc2354 service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:15.090472 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b571685-b695-4a36-aeeb-50df29cc2354 service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:15.090983 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b571685-b695-4a36-aeeb-50df29cc2354 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:15.093373 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-5b571685-b695-4a36-aeeb-50df29cc2354 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:04:15.108363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b571685-b695-4a36-aeeb-50df29cc2354 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:15.109946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b571685-b695-4a36-aeeb-50df29cc2354 service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:15.131637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1341/2725] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:04:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:04:16.560506 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=26 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:04:18.135716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5424c7d-c172-4f69-92c1-79ec2ca104f2 service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:18.150297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5424c7d-c172-4f69-92c1-79ec2ca104f2 service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:18.158423 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0689c-d97c-4eb7-9234-b220fabd4707 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:18.159143 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0689c-d97c-4eb7-9234-b220fabd4707 service nova] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:18.159358 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0689c-d97c-4eb7-9234-b220fabd4707 service nova] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:18.159711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0689c-d97c-4eb7-9234-b220fabd4707 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:18.161627 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-3ec0689c-d97c-4eb7-9234-b220fabd4707 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:04:18.176346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0689c-d97c-4eb7-9234-b220fabd4707 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:18.177921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0689c-d97c-4eb7-9234-b220fabd4707 service nova] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:18.190900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1385/2726] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:04:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:04:25.117958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=6 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:04:26.040193 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b571685-b695-4a36-aeeb-50df29cc2354 service nova] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:26.046448 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-5b571685-b695-4a36-aeeb-50df29cc2354 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:04:26.116844 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b571685-b695-4a36-aeeb-50df29cc2354 service nova] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:26.121743 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c44444-7d9c-4f55-9a49-b32c33a3c6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:26.122417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c44444-7d9c-4f55-9a49-b32c33a3c6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:26.122930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c44444-7d9c-4f55-9a49-b32c33a3c6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:26.123208 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c44444-7d9c-4f55-9a49-b32c33a3c6a9 admin admin] PATH_INFO: `/v3/users/fd489ef668494ec98cf9d167a1ded3ce` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:26.124486 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c44444-7d9c-4f55-9a49-b32c33a3c6a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd489ef668494ec98cf9d167a1ded3ce)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:26.126249 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c44444-7d9c-4f55-9a49-b32c33a3c6a9 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:26.175395 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-98c44444-7d9c-4f55-9a49-b32c33a3c6a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd489ef668494ec98cf9d167a1ded3ce'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:04:26.187847 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-98c44444-7d9c-4f55-9a49-b32c33a3c6a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd489ef668494ec98cf9d167a1ded3ce'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:04:26.197150 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-98c44444-7d9c-4f55-9a49-b32c33a3c6a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd489ef668494ec98cf9d167a1ded3ce'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:04:26.197826 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-98c44444-7d9c-4f55-9a49-b32c33a3c6a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd489ef668494ec98cf9d167a1ded3ce'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:04:26.205459 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-98c44444-7d9c-4f55-9a49-b32c33a3c6a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd489ef668494ec98cf9d167a1ded3ce'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:04:26.212186 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1342/2727] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:04:26 2022] DELETE /identity/v3/users/fd489ef668494ec98cf9d167a1ded3ce => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:04:26.386932 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0689c-d97c-4eb7-9234-b220fabd4707 service nova] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:26.401755 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0689c-d97c-4eb7-9234-b220fabd4707 service nova] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:26.464855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f7494b-3cef-4a33-b109-9e148e3da746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:26.465513 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7494b-3cef-4a33-b109-9e148e3da746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:26.465796 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7494b-3cef-4a33-b109-9e148e3da746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:26.466070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f7494b-3cef-4a33-b109-9e148e3da746 admin admin] PATH_INFO: `/v3/projects/8ed92e8abb0c4adf85c6854934423f1e` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:26.467318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f7494b-3cef-4a33-b109-9e148e3da746 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ed92e8abb0c4adf85c6854934423f1e)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:26.469213 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f7494b-3cef-4a33-b109-9e148e3da746 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:26.518808 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-91f7494b-3cef-4a33-b109-9e148e3da746 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ed92e8abb0c4adf85c6854934423f1e', 'request_id': 'req-91f7494b-3cef-4a33-b109-9e148e3da746'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:04:26.527578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-91f7494b-3cef-4a33-b109-9e148e3da746 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ed92e8abb0c4adf85c6854934423f1e', 'request_id': 'req-91f7494b-3cef-4a33-b109-9e148e3da746'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:04:26.534455 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-91f7494b-3cef-4a33-b109-9e148e3da746 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ed92e8abb0c4adf85c6854934423f1e', 'request_id': 'req-91f7494b-3cef-4a33-b109-9e148e3da746'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:04:26.545694 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-91f7494b-3cef-4a33-b109-9e148e3da746 admin admin] The token cache is being invalidate because project 8ed92e8abb0c4adf85c6854934423f1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:04:26.545984 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-91f7494b-3cef-4a33-b109-9e148e3da746 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:04:26.546675 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-91f7494b-3cef-4a33-b109-9e148e3da746 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:04:26.548402 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1386/2728] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:04:26 2022] DELETE /identity/v3/projects/8ed92e8abb0c4adf85c6854934423f1e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:04:26.553121 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cbdd8-661b-4852-8449-b75be305ccba None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:26.553482 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cbdd8-661b-4852-8449-b75be305ccba None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:26.553924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cbdd8-661b-4852-8449-b75be305ccba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:26.573841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-5a2cbdd8-661b-4852-8449-b75be305ccba None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:04:26.575135 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-5a2cbdd8-661b-4852-8449-b75be305ccba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:04:26.640477 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1343/2729] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:04:26.649729 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab95e8-ecc4-487e-b1d7-68f6b05c9457 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:26.650011 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab95e8-ecc4-487e-b1d7-68f6b05c9457 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:26.650278 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab95e8-ecc4-487e-b1d7-68f6b05c9457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:26.669540 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-bfab95e8-ecc4-487e-b1d7-68f6b05c9457 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:04:26.670765 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-bfab95e8-ecc4-487e-b1d7-68f6b05c9457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:04:26.678804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1387/2730] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:04:26.682851 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cbdd8-661b-4852-8449-b75be305ccba None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:26.694385 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cbdd8-661b-4852-8449-b75be305ccba None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:26.699288 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05be54d-8f80-47e4-9649-50dfe0593a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:26.699917 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05be54d-8f80-47e4-9649-50dfe0593a88 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:26.700200 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05be54d-8f80-47e4-9649-50dfe0593a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:26.700473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05be54d-8f80-47e4-9649-50dfe0593a88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:26.701169 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05be54d-8f80-47e4-9649-50dfe0593a88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 13:04:26.701473 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05be54d-8f80-47e4-9649-50dfe0593a88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:26.703084 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05be54d-8f80-47e4-9649-50dfe0593a88 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:26.715046 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1344/2731] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 13:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 13:04:26.719265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab95e8-ecc4-487e-b1d7-68f6b05c9457 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:26.728498 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab95e8-ecc4-487e-b1d7-68f6b05c9457 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:26.733289 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f221c-cceb-4751-b61a-873369a41f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:26.733894 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f221c-cceb-4751-b61a-873369a41f40 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:26.734172 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f221c-cceb-4751-b61a-873369a41f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:26.734442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f221c-cceb-4751-b61a-873369a41f40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:26.735149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f221c-cceb-4751-b61a-873369a41f40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:26.737587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84f221c-cceb-4751-b61a-873369a41f40 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:26.773677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1388/2732] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:04:26 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:04:26.777698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05be54d-8f80-47e4-9649-50dfe0593a88 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:26.787711 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05be54d-8f80-47e4-9649-50dfe0593a88 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:26.791331 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:04:26.844873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7d009-54bf-415b-bbaa-7232799b5a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:26.845463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7d009-54bf-415b-bbaa-7232799b5a27 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:26.845744 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7d009-54bf-415b-bbaa-7232799b5a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:26.846017 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e7d009-54bf-415b-bbaa-7232799b5a27 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:26.846712 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7d009-54bf-415b-bbaa-7232799b5a27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:26.848604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e7d009-54bf-415b-bbaa-7232799b5a27 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:26.870130 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1345/2733] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:04:26 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:04:26.874258 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f221c-cceb-4751-b61a-873369a41f40 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:26.888339 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f221c-cceb-4751-b61a-873369a41f40 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:26.893082 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305266b1-a421-41ef-970e-6d09b3baf9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:26.893667 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305266b1-a421-41ef-970e-6d09b3baf9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:26.893947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305266b1-a421-41ef-970e-6d09b3baf9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:26.894221 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305266b1-a421-41ef-970e-6d09b3baf9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:26.894959 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305266b1-a421-41ef-970e-6d09b3baf9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:26.896655 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305266b1-a421-41ef-970e-6d09b3baf9d0 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:26.906443 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1389/2734] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 13:04:26.910449 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7d009-54bf-415b-bbaa-7232799b5a27 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:26.920589 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e7d009-54bf-415b-bbaa-7232799b5a27 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:26.925718 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8c0d3-44c1-4f8c-b674-59f397aedd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:26.926313 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8c0d3-44c1-4f8c-b674-59f397aedd8b admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:26.926636 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8c0d3-44c1-4f8c-b674-59f397aedd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:26.926913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8c0d3-44c1-4f8c-b674-59f397aedd8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:26.927605 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8c0d3-44c1-4f8c-b674-59f397aedd8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:26.929182 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8c0d3-44c1-4f8c-b674-59f397aedd8b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:26.939870 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1346/2735] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:04:26.944353 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305266b1-a421-41ef-970e-6d09b3baf9d0 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:26.954573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305266b1-a421-41ef-970e-6d09b3baf9d0 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:26.959119 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194f200-329b-4c7f-afdf-a22a0d84646e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:26.959769 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194f200-329b-4c7f-afdf-a22a0d84646e admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:26.960046 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194f200-329b-4c7f-afdf-a22a0d84646e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:26.960319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d194f200-329b-4c7f-afdf-a22a0d84646e admin admin] PATH_INFO: `/v3/projects/1cad6cb016624de59863e316031937a8/users/6132806865da4ba98b96fb0299bdb86f/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:26.975023 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194f200-329b-4c7f-afdf-a22a0d84646e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cad6cb016624de59863e316031937a8, user_id=6132806865da4ba98b96fb0299bdb86f, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:26.977638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d194f200-329b-4c7f-afdf-a22a0d84646e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:26.991342 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1390/2736] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:04:26 2022] PUT /identity/v3/projects/1cad6cb016624de59863e316031937a8/users/6132806865da4ba98b96fb0299bdb86f/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:04:27.001454 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3209434-7615-4a8a-a59c-8757b5345455 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:27.001738 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3209434-7615-4a8a-a59c-8757b5345455 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:27.002014 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3209434-7615-4a8a-a59c-8757b5345455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:27.029130 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-e3209434-7615-4a8a-a59c-8757b5345455 None None] MFA Rules not processed for user `6132806865da4ba98b96fb0299bdb86f`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:04:27.030335 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e3209434-7615-4a8a-a59c-8757b5345455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:04:27.095393 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1347/2737] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:04:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:04:27.099872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194f200-329b-4c7f-afdf-a22a0d84646e admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:27.105529 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:04:27.178090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d194f200-329b-4c7f-afdf-a22a0d84646e admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:27.187187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb17e0-c82b-441c-a192-b68f5cab5a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:27.187855 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb17e0-c82b-441c-a192-b68f5cab5a3e admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:27.188136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb17e0-c82b-441c-a192-b68f5cab5a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:27.188405 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb17e0-c82b-441c-a192-b68f5cab5a3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:27.189833 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb17e0-c82b-441c-a192-b68f5cab5a3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:27.192358 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb17e0-c82b-441c-a192-b68f5cab5a3e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:27.226523 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1391/2738] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:04:27 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 13:04:27.230398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3209434-7615-4a8a-a59c-8757b5345455 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:27.242741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3209434-7615-4a8a-a59c-8757b5345455 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:27.304615 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a080c9-ab63-4589-94f3-a08b313bb326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:27.305252 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a080c9-ab63-4589-94f3-a08b313bb326 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:27.305522 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a080c9-ab63-4589-94f3-a08b313bb326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:27.305793 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a080c9-ab63-4589-94f3-a08b313bb326 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:27.306464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a080c9-ab63-4589-94f3-a08b313bb326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:27.308387 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a080c9-ab63-4589-94f3-a08b313bb326 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:27.328705 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1348/2739] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:04:27 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:04:27.332553 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb17e0-c82b-441c-a192-b68f5cab5a3e admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:27.344188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb17e0-c82b-441c-a192-b68f5cab5a3e admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:27.350347 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632334e1-27c4-4271-8213-7e14fac3e71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:27.351014 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632334e1-27c4-4271-8213-7e14fac3e71b admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:27.351435 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632334e1-27c4-4271-8213-7e14fac3e71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:27.351579 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632334e1-27c4-4271-8213-7e14fac3e71b admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:27.352244 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632334e1-27c4-4271-8213-7e14fac3e71b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:27.353844 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632334e1-27c4-4271-8213-7e14fac3e71b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:27.365075 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1392/2740] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:04:27.369438 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a080c9-ab63-4589-94f3-a08b313bb326 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:27.375883 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:04:27.379827 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a080c9-ab63-4589-94f3-a08b313bb326 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:27.385621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e72bef-9d50-4ae7-81a4-65c46dd8b002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:27.386300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e72bef-9d50-4ae7-81a4-65c46dd8b002 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:27.386616 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e72bef-9d50-4ae7-81a4-65c46dd8b002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:27.386897 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e72bef-9d50-4ae7-81a4-65c46dd8b002 admin admin] PATH_INFO: `/v3/projects/d438df51642741308bbeb45bcf325bb8/users/53325a5de2ff4f8aa6c0d6acb1401d7e/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:27.402146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e72bef-9d50-4ae7-81a4-65c46dd8b002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d438df51642741308bbeb45bcf325bb8, user_id=53325a5de2ff4f8aa6c0d6acb1401d7e, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:27.404763 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e72bef-9d50-4ae7-81a4-65c46dd8b002 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:27.419898 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1349/2741] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:04:27 2022] PUT /identity/v3/projects/d438df51642741308bbeb45bcf325bb8/users/53325a5de2ff4f8aa6c0d6acb1401d7e/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:04:27.423911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632334e1-27c4-4271-8213-7e14fac3e71b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:27.434829 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632334e1-27c4-4271-8213-7e14fac3e71b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:27.491724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b22326-e150-4b28-be53-8f80801c9e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:27.492341 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b22326-e150-4b28-be53-8f80801c9e22 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:27.492626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b22326-e150-4b28-be53-8f80801c9e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:27.492903 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b22326-e150-4b28-be53-8f80801c9e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:27.493595 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b22326-e150-4b28-be53-8f80801c9e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:27.495230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b22326-e150-4b28-be53-8f80801c9e22 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:27.504830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1393/2742] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:04:27.508953 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e72bef-9d50-4ae7-81a4-65c46dd8b002 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:27.519667 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e72bef-9d50-4ae7-81a4-65c46dd8b002 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:27.524195 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dac5b-064e-40d2-a5bd-d2af3158735a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:27.524850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dac5b-064e-40d2-a5bd-d2af3158735a admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:27.525133 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dac5b-064e-40d2-a5bd-d2af3158735a admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:27.525404 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dac5b-064e-40d2-a5bd-d2af3158735a admin admin] PATH_INFO: `/v3/projects/d438df51642741308bbeb45bcf325bb8/users/53325a5de2ff4f8aa6c0d6acb1401d7e/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:27.527397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7dac5b-064e-40d2-a5bd-d2af3158735a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d438df51642741308bbeb45bcf325bb8, user_id=53325a5de2ff4f8aa6c0d6acb1401d7e, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:27.530009 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7dac5b-064e-40d2-a5bd-d2af3158735a admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:27.540572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1350/2743] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:04:27 2022] PUT /identity/v3/projects/d438df51642741308bbeb45bcf325bb8/users/53325a5de2ff4f8aa6c0d6acb1401d7e/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:04:27.550753 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479422d4-b94a-4475-942a-43eab7927c99 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:27.551045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479422d4-b94a-4475-942a-43eab7927c99 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:27.551324 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479422d4-b94a-4475-942a-43eab7927c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:27.578889 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-479422d4-b94a-4475-942a-43eab7927c99 None None] MFA Rules not processed for user `53325a5de2ff4f8aa6c0d6acb1401d7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:04:27.580093 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-479422d4-b94a-4475-942a-43eab7927c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:04:27.648264 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:04:27.654745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1394/2744] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:04:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:04:27.664945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dac5b-064e-40d2-a5bd-d2af3158735a admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:27.666811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-4d7dac5b-064e-40d2-a5bd-d2af3158735a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:04:27.725982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dac5b-064e-40d2-a5bd-d2af3158735a admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:27.729645 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92f21b-3c70-43f7-8dcf-886f99a67407 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:27.730258 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92f21b-3c70-43f7-8dcf-886f99a67407 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:27.730571 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92f21b-3c70-43f7-8dcf-886f99a67407 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:27.730843 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92f21b-3c70-43f7-8dcf-886f99a67407 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:27.739608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92f21b-3c70-43f7-8dcf-886f99a67407 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:27.741022 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92f21b-3c70-43f7-8dcf-886f99a67407 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:27.750250 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1351/2745] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:04:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:04:27.822265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479422d4-b94a-4475-942a-43eab7927c99 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:04:27.832583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479422d4-b94a-4475-942a-43eab7927c99 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:04:27.836233 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472660a4-77ef-4511-8a23-7b0e6ac0b1b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:04:27.836840 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472660a4-77ef-4511-8a23-7b0e6ac0b1b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:04:27.837148 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472660a4-77ef-4511-8a23-7b0e6ac0b1b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:04:27.837425 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472660a4-77ef-4511-8a23-7b0e6ac0b1b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:04:27.893057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472660a4-77ef-4511-8a23-7b0e6ac0b1b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:04:27.895205 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472660a4-77ef-4511-8a23-7b0e6ac0b1b4 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:04:27.904947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1395/2746] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:04:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:04:37.746163 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=39,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:04:37.900026 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=28 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:06.033130 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92f21b-3c70-43f7-8dcf-886f99a67407 service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:06.098978 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92f21b-3c70-43f7-8dcf-886f99a67407 service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:06.103479 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcf8b6-a884-49d6-9795-2a1c3a08080d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:06.104106 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcf8b6-a884-49d6-9795-2a1c3a08080d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:06.104385 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcf8b6-a884-49d6-9795-2a1c3a08080d admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:06.104655 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabcf8b6-a884-49d6-9795-2a1c3a08080d admin admin] PATH_INFO: `/v3/users/6132806865da4ba98b96fb0299bdb86f` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:06.105810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcf8b6-a884-49d6-9795-2a1c3a08080d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6132806865da4ba98b96fb0299bdb86f)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:06.107724 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabcf8b6-a884-49d6-9795-2a1c3a08080d admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:06.152127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-eabcf8b6-a884-49d6-9795-2a1c3a08080d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6132806865da4ba98b96fb0299bdb86f'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:06.164461 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-eabcf8b6-a884-49d6-9795-2a1c3a08080d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6132806865da4ba98b96fb0299bdb86f'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:06.171939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-eabcf8b6-a884-49d6-9795-2a1c3a08080d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6132806865da4ba98b96fb0299bdb86f'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:06.172556 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-eabcf8b6-a884-49d6-9795-2a1c3a08080d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6132806865da4ba98b96fb0299bdb86f'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:06.180162 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-eabcf8b6-a884-49d6-9795-2a1c3a08080d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6132806865da4ba98b96fb0299bdb86f'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:06.185849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1352/2747] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:05:06 2022] DELETE /identity/v3/users/6132806865da4ba98b96fb0299bdb86f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:06.190408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5f4c1-f41a-41e6-ba7a-8c930da01fa1 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:06.190913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5f4c1-f41a-41e6-ba7a-8c930da01fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:06.191016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5f4c1-f41a-41e6-ba7a-8c930da01fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:06.209296 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-82b5f4c1-f41a-41e6-ba7a-8c930da01fa1 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:06.210541 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-82b5f4c1-f41a-41e6-ba7a-8c930da01fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:06.267442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1396/2748] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:06.277033 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcf8b6-a884-49d6-9795-2a1c3a08080d admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:06.278436 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-eabcf8b6-a884-49d6-9795-2a1c3a08080d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:06.333742 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabcf8b6-a884-49d6-9795-2a1c3a08080d admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:06.337345 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68ab8a-b2ec-41df-aee1-e94b9f02fe4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:06.337947 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68ab8a-b2ec-41df-aee1-e94b9f02fe4c service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:06.338222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68ab8a-b2ec-41df-aee1-e94b9f02fe4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:06.338505 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68ab8a-b2ec-41df-aee1-e94b9f02fe4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:06.347271 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68ab8a-b2ec-41df-aee1-e94b9f02fe4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:06.349212 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68ab8a-b2ec-41df-aee1-e94b9f02fe4c service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:06.358706 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1353/2749] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:06.619583 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5f4c1-f41a-41e6-ba7a-8c930da01fa1 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:06.621377 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-82b5f4c1-f41a-41e6-ba7a-8c930da01fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:06.635395 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5f4c1-f41a-41e6-ba7a-8c930da01fa1 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:06.640422 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550ad9f-d60a-4b0d-af93-4f38fd01045b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:06.641067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550ad9f-d60a-4b0d-af93-4f38fd01045b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:06.641345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550ad9f-d60a-4b0d-af93-4f38fd01045b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:06.641616 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b550ad9f-d60a-4b0d-af93-4f38fd01045b admin admin] PATH_INFO: `/v3/projects/1cad6cb016624de59863e316031937a8` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:06.643246 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550ad9f-d60a-4b0d-af93-4f38fd01045b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cad6cb016624de59863e316031937a8)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:06.645945 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b550ad9f-d60a-4b0d-af93-4f38fd01045b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:06.695263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-b550ad9f-d60a-4b0d-af93-4f38fd01045b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cad6cb016624de59863e316031937a8', 'request_id': 'req-b550ad9f-d60a-4b0d-af93-4f38fd01045b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:06.702629 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-b550ad9f-d60a-4b0d-af93-4f38fd01045b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cad6cb016624de59863e316031937a8', 'request_id': 'req-b550ad9f-d60a-4b0d-af93-4f38fd01045b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:06.709170 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-b550ad9f-d60a-4b0d-af93-4f38fd01045b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cad6cb016624de59863e316031937a8', 'request_id': 'req-b550ad9f-d60a-4b0d-af93-4f38fd01045b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:06.719083 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-b550ad9f-d60a-4b0d-af93-4f38fd01045b admin admin] The token cache is being invalidate because project 1cad6cb016624de59863e316031937a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:05:06.719366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-b550ad9f-d60a-4b0d-af93-4f38fd01045b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:06.719979 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-b550ad9f-d60a-4b0d-af93-4f38fd01045b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:06.721669 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1397/2750] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:05:06 2022] DELETE /identity/v3/projects/1cad6cb016624de59863e316031937a8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:06.725975 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68ab8a-b2ec-41df-aee1-e94b9f02fe4c service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:06.735888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68ab8a-b2ec-41df-aee1-e94b9f02fe4c service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:06.753803 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:06.793948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025fce5-4736-4fa7-bc93-9ac69a48c023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:06.794613 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025fce5-4736-4fa7-bc93-9ac69a48c023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:06.794893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025fce5-4736-4fa7-bc93-9ac69a48c023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:06.795161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025fce5-4736-4fa7-bc93-9ac69a48c023 admin admin] PATH_INFO: `/v3/users/53325a5de2ff4f8aa6c0d6acb1401d7e` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:06.796306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0025fce5-4736-4fa7-bc93-9ac69a48c023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53325a5de2ff4f8aa6c0d6acb1401d7e)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:06.798183 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0025fce5-4736-4fa7-bc93-9ac69a48c023 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:06.842904 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0025fce5-4736-4fa7-bc93-9ac69a48c023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53325a5de2ff4f8aa6c0d6acb1401d7e'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:06.854457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0025fce5-4736-4fa7-bc93-9ac69a48c023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53325a5de2ff4f8aa6c0d6acb1401d7e'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:06.861807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0025fce5-4736-4fa7-bc93-9ac69a48c023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53325a5de2ff4f8aa6c0d6acb1401d7e'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:06.862417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0025fce5-4736-4fa7-bc93-9ac69a48c023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53325a5de2ff4f8aa6c0d6acb1401d7e'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:06.869341 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-0025fce5-4736-4fa7-bc93-9ac69a48c023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53325a5de2ff4f8aa6c0d6acb1401d7e'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:06.875538 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1354/2751] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:05:06 2022] DELETE /identity/v3/users/53325a5de2ff4f8aa6c0d6acb1401d7e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:07.139466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550ad9f-d60a-4b0d-af93-4f38fd01045b admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:07.141070 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b550ad9f-d60a-4b0d-af93-4f38fd01045b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:07.154960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b550ad9f-d60a-4b0d-af93-4f38fd01045b admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:07.215843 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588f1a34-bb3d-446f-91bb-c72d67ce389b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:07.216496 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f1a34-bb3d-446f-91bb-c72d67ce389b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:07.216746 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f1a34-bb3d-446f-91bb-c72d67ce389b admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:07.217015 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f1a34-bb3d-446f-91bb-c72d67ce389b admin admin] PATH_INFO: `/v3/projects/d438df51642741308bbeb45bcf325bb8` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:07.218144 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f1a34-bb3d-446f-91bb-c72d67ce389b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d438df51642741308bbeb45bcf325bb8)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:07.220054 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588f1a34-bb3d-446f-91bb-c72d67ce389b admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:07.249062 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:07.266269 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-588f1a34-bb3d-446f-91bb-c72d67ce389b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd438df51642741308bbeb45bcf325bb8', 'request_id': 'req-588f1a34-bb3d-446f-91bb-c72d67ce389b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:07.273458 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-588f1a34-bb3d-446f-91bb-c72d67ce389b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd438df51642741308bbeb45bcf325bb8', 'request_id': 'req-588f1a34-bb3d-446f-91bb-c72d67ce389b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:07.279796 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-588f1a34-bb3d-446f-91bb-c72d67ce389b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd438df51642741308bbeb45bcf325bb8', 'request_id': 'req-588f1a34-bb3d-446f-91bb-c72d67ce389b'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:07.290154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-588f1a34-bb3d-446f-91bb-c72d67ce389b admin admin] The token cache is being invalidate because project d438df51642741308bbeb45bcf325bb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:05:07.290422 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-588f1a34-bb3d-446f-91bb-c72d67ce389b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:07.291068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-588f1a34-bb3d-446f-91bb-c72d67ce389b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:07.292820 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1398/2752] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:05:07 2022] DELETE /identity/v3/projects/d438df51642741308bbeb45bcf325bb8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:07.297758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867c7ec-c85d-4291-a7a9-cdae58235ee2 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:07.298216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867c7ec-c85d-4291-a7a9-cdae58235ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:07.298340 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867c7ec-c85d-4291-a7a9-cdae58235ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:07.318330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-f867c7ec-c85d-4291-a7a9-cdae58235ee2 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:07.319543 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f867c7ec-c85d-4291-a7a9-cdae58235ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:07.379836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1355/2753] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:07.384507 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc9dd4-07a4-4db6-9f6e-6e091c585dc7 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:07.384794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc9dd4-07a4-4db6-9f6e-6e091c585dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:07.385071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc9dd4-07a4-4db6-9f6e-6e091c585dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:07.404582 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-38dc9dd4-07a4-4db6-9f6e-6e091c585dc7 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:07.405790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-38dc9dd4-07a4-4db6-9f6e-6e091c585dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:07.413826 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1399/2754] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:07.418336 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867c7ec-c85d-4291-a7a9-cdae58235ee2 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:07.428439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867c7ec-c85d-4291-a7a9-cdae58235ee2 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:07.433040 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cecc17-08ad-451a-82f1-b2567e96952a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:07.433672 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cecc17-08ad-451a-82f1-b2567e96952a admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:07.433951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cecc17-08ad-451a-82f1-b2567e96952a admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:07.434222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cecc17-08ad-451a-82f1-b2567e96952a admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:07.434951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cecc17-08ad-451a-82f1-b2567e96952a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 13:05:07.435255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cecc17-08ad-451a-82f1-b2567e96952a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:07.436865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cecc17-08ad-451a-82f1-b2567e96952a admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:07.448767 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1356/2755] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 13:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 03 13:05:07.452894 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc9dd4-07a4-4db6-9f6e-6e091c585dc7 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:07.462114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc9dd4-07a4-4db6-9f6e-6e091c585dc7 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:07.466731 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f39ca-1437-4178-8967-f1ea98bbdc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:07.467320 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f39ca-1437-4178-8967-f1ea98bbdc62 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:07.467600 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f39ca-1437-4178-8967-f1ea98bbdc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:07.467865 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f39ca-1437-4178-8967-f1ea98bbdc62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:07.468517 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f39ca-1437-4178-8967-f1ea98bbdc62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:07.470384 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f39ca-1437-4178-8967-f1ea98bbdc62 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:07.498178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1400/2756] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:05:07 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:07.502359 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cecc17-08ad-451a-82f1-b2567e96952a admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:07.513055 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cecc17-08ad-451a-82f1-b2567e96952a admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:07.538049 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:07.570110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568031a-8850-4821-a0df-a6ed1c7450da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:07.570744 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568031a-8850-4821-a0df-a6ed1c7450da admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:07.571028 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568031a-8850-4821-a0df-a6ed1c7450da admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:07.571303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568031a-8850-4821-a0df-a6ed1c7450da admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:07.571957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568031a-8850-4821-a0df-a6ed1c7450da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:07.573852 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6568031a-8850-4821-a0df-a6ed1c7450da admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:07.595276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1357/2757] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:05:07 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:07.599847 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f39ca-1437-4178-8967-f1ea98bbdc62 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:07.610071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f39ca-1437-4178-8967-f1ea98bbdc62 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:07.615239 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d18f3-401d-4fcf-8a87-ec57347c1144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:07.615846 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d18f3-401d-4fcf-8a87-ec57347c1144 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:07.616125 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d18f3-401d-4fcf-8a87-ec57347c1144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:07.616396 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d18f3-401d-4fcf-8a87-ec57347c1144 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:07.617087 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d18f3-401d-4fcf-8a87-ec57347c1144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:07.618716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d18f3-401d-4fcf-8a87-ec57347c1144 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:07.628343 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1401/2758] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:07.632008 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568031a-8850-4821-a0df-a6ed1c7450da admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:07.642011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568031a-8850-4821-a0df-a6ed1c7450da admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:07.646727 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6caeed-ab5d-405b-a721-e4dc32370b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:07.647337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6caeed-ab5d-405b-a721-e4dc32370b03 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:07.647618 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6caeed-ab5d-405b-a721-e4dc32370b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:07.647895 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6caeed-ab5d-405b-a721-e4dc32370b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:07.648586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6caeed-ab5d-405b-a721-e4dc32370b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:07.650171 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6caeed-ab5d-405b-a721-e4dc32370b03 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:07.661201 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1358/2759] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:07.665274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d18f3-401d-4fcf-8a87-ec57347c1144 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:07.675518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d18f3-401d-4fcf-8a87-ec57347c1144 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:07.680755 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed2c69-1c54-43cd-8f90-529d67445680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:07.681434 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2c69-1c54-43cd-8f90-529d67445680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:07.681711 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2c69-1c54-43cd-8f90-529d67445680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:07.681985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed2c69-1c54-43cd-8f90-529d67445680 admin admin] PATH_INFO: `/v3/projects/999c035701044c69bd451debea36c579/users/844dbb794c95487cb2635395952db9da/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:07.699673 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed2c69-1c54-43cd-8f90-529d67445680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=999c035701044c69bd451debea36c579, user_id=844dbb794c95487cb2635395952db9da, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:07.702308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed2c69-1c54-43cd-8f90-529d67445680 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:07.714522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1402/2760] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:05:07 2022] PUT /identity/v3/projects/999c035701044c69bd451debea36c579/users/844dbb794c95487cb2635395952db9da/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:07.726255 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10434cf1-0644-4a3e-ab2f-b795488b065f None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:07.726563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10434cf1-0644-4a3e-ab2f-b795488b065f None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:07.726836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10434cf1-0644-4a3e-ab2f-b795488b065f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:07.754665 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-10434cf1-0644-4a3e-ab2f-b795488b065f None None] MFA Rules not processed for user `844dbb794c95487cb2635395952db9da`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:07.755896 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-10434cf1-0644-4a3e-ab2f-b795488b065f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:07.818899 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1359/2761] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:07.823363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed2c69-1c54-43cd-8f90-529d67445680 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:07.888724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed2c69-1c54-43cd-8f90-529d67445680 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:07.893373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195c25c-0a94-4c7c-b9dc-58c315972d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:07.893989 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195c25c-0a94-4c7c-b9dc-58c315972d42 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:07.894269 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195c25c-0a94-4c7c-b9dc-58c315972d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:07.894573 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195c25c-0a94-4c7c-b9dc-58c315972d42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:07.895262 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195c25c-0a94-4c7c-b9dc-58c315972d42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:07.897165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195c25c-0a94-4c7c-b9dc-58c315972d42 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:07.923545 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1403/2762] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:05:07 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:07.928148 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10434cf1-0644-4a3e-ab2f-b795488b065f None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:07.938549 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10434cf1-0644-4a3e-ab2f-b795488b065f None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:07.993664 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4255bd-4273-42d9-89e9-eb2080b503f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:07.994272 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4255bd-4273-42d9-89e9-eb2080b503f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:07.994575 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4255bd-4273-42d9-89e9-eb2080b503f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:07.994846 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4255bd-4273-42d9-89e9-eb2080b503f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:07.995506 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4255bd-4273-42d9-89e9-eb2080b503f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:07.997395 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4255bd-4273-42d9-89e9-eb2080b503f8 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:08.017652 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1360/2763] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:05:07 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:08.022224 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195c25c-0a94-4c7c-b9dc-58c315972d42 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:08.041189 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195c25c-0a94-4c7c-b9dc-58c315972d42 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:08.053572 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320becf2-86a8-4ef9-9114-61921c2dd055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:08.054178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320becf2-86a8-4ef9-9114-61921c2dd055 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:08.054457 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320becf2-86a8-4ef9-9114-61921c2dd055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:08.054756 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320becf2-86a8-4ef9-9114-61921c2dd055 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:08.055441 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320becf2-86a8-4ef9-9114-61921c2dd055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:08.057016 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320becf2-86a8-4ef9-9114-61921c2dd055 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:08.067718 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1404/2764] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:08.071805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4255bd-4273-42d9-89e9-eb2080b503f8 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:08.081680 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4255bd-4273-42d9-89e9-eb2080b503f8 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:08.086758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d083acb-b6fc-4caf-b4e8-5b09f92a5423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:08.087422 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d083acb-b6fc-4caf-b4e8-5b09f92a5423 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:08.087691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d083acb-b6fc-4caf-b4e8-5b09f92a5423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:08.087963 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d083acb-b6fc-4caf-b4e8-5b09f92a5423 admin admin] PATH_INFO: `/v3/projects/16b8432656cb474f807d0ab17341f3f4/users/daf2346f3e0049cfb9850a079b33dc6c/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:08.102980 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d083acb-b6fc-4caf-b4e8-5b09f92a5423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16b8432656cb474f807d0ab17341f3f4, user_id=daf2346f3e0049cfb9850a079b33dc6c, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:08.105592 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d083acb-b6fc-4caf-b4e8-5b09f92a5423 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:08.118537 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1361/2765] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:05:08 2022] PUT /identity/v3/projects/16b8432656cb474f807d0ab17341f3f4/users/daf2346f3e0049cfb9850a079b33dc6c/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:08.122293 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320becf2-86a8-4ef9-9114-61921c2dd055 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:08.132122 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320becf2-86a8-4ef9-9114-61921c2dd055 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:08.151943 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:08.190265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264e0b1-167f-4e0b-8d77-ca1e657df5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:08.190898 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264e0b1-167f-4e0b-8d77-ca1e657df5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:08.191173 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264e0b1-167f-4e0b-8d77-ca1e657df5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:08.191444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264e0b1-167f-4e0b-8d77-ca1e657df5b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:08.192120 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264e0b1-167f-4e0b-8d77-ca1e657df5b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:08.193700 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264e0b1-167f-4e0b-8d77-ca1e657df5b8 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:08.203728 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1405/2766] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:08.208430 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d083acb-b6fc-4caf-b4e8-5b09f92a5423 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:08.213876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:08.220031 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d083acb-b6fc-4caf-b4e8-5b09f92a5423 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:08.225444 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b64cb8-996e-4307-a91f-72a6905029c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:08.226094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b64cb8-996e-4307-a91f-72a6905029c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:08.226372 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b64cb8-996e-4307-a91f-72a6905029c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:08.226670 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b64cb8-996e-4307-a91f-72a6905029c5 admin admin] PATH_INFO: `/v3/projects/16b8432656cb474f807d0ab17341f3f4/users/daf2346f3e0049cfb9850a079b33dc6c/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:08.228594 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b64cb8-996e-4307-a91f-72a6905029c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16b8432656cb474f807d0ab17341f3f4, user_id=daf2346f3e0049cfb9850a079b33dc6c, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:08.231257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b64cb8-996e-4307-a91f-72a6905029c5 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:08.241119 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1362/2767] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:05:08 2022] PUT /identity/v3/projects/16b8432656cb474f807d0ab17341f3f4/users/daf2346f3e0049cfb9850a079b33dc6c/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:08.305519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3c391-9c5d-4d04-b100-be6a6437a719 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:08.306049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3c391-9c5d-4d04-b100-be6a6437a719 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:08.306049 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3c391-9c5d-4d04-b100-be6a6437a719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:08.334055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-a4a3c391-9c5d-4d04-b100-be6a6437a719 None None] MFA Rules not processed for user `daf2346f3e0049cfb9850a079b33dc6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:08.335322 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a4a3c391-9c5d-4d04-b100-be6a6437a719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:08.401210 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1406/2768] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:08.412139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b64cb8-996e-4307-a91f-72a6905029c5 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:08.414212 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d6b64cb8-996e-4307-a91f-72a6905029c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:08.469510 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b64cb8-996e-4307-a91f-72a6905029c5 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:08.473079 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039789d1-85dc-442a-9d31-a9df93a515b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:08.473673 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039789d1-85dc-442a-9d31-a9df93a515b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:08.473949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039789d1-85dc-442a-9d31-a9df93a515b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:08.474215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039789d1-85dc-442a-9d31-a9df93a515b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:08.482956 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039789d1-85dc-442a-9d31-a9df93a515b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:08.484352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039789d1-85dc-442a-9d31-a9df93a515b2 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:08.493451 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1363/2769] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:08 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:08.572391 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3c391-9c5d-4d04-b100-be6a6437a719 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:08.581941 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3c391-9c5d-4d04-b100-be6a6437a719 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:08.586137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a20dd6-6d6e-4812-9ddb-e995104efd51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:08.586777 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a20dd6-6d6e-4812-9ddb-e995104efd51 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:08.587061 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a20dd6-6d6e-4812-9ddb-e995104efd51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:08.587336 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a20dd6-6d6e-4812-9ddb-e995104efd51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:08.643862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a20dd6-6d6e-4812-9ddb-e995104efd51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:08.645904 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a20dd6-6d6e-4812-9ddb-e995104efd51 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:08.654894 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1407/2770] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:17.742122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039789d1-85dc-442a-9d31-a9df93a515b2 service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:17.802582 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039789d1-85dc-442a-9d31-a9df93a515b2 service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:17.807119 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638ebf2-40e4-4636-b090-e7eefc9f26c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:17.807750 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638ebf2-40e4-4636-b090-e7eefc9f26c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:17.808031 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638ebf2-40e4-4636-b090-e7eefc9f26c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:17.808311 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8638ebf2-40e4-4636-b090-e7eefc9f26c0 admin admin] PATH_INFO: `/v3/users/844dbb794c95487cb2635395952db9da` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:17.809622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8638ebf2-40e4-4636-b090-e7eefc9f26c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=844dbb794c95487cb2635395952db9da)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:17.811566 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8638ebf2-40e4-4636-b090-e7eefc9f26c0 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:17.856658 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8638ebf2-40e4-4636-b090-e7eefc9f26c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '844dbb794c95487cb2635395952db9da'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:17.868377 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8638ebf2-40e4-4636-b090-e7eefc9f26c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '844dbb794c95487cb2635395952db9da'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:17.881449 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8638ebf2-40e4-4636-b090-e7eefc9f26c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '844dbb794c95487cb2635395952db9da'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:17.882071 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8638ebf2-40e4-4636-b090-e7eefc9f26c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '844dbb794c95487cb2635395952db9da'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:17.889179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8638ebf2-40e4-4636-b090-e7eefc9f26c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '844dbb794c95487cb2635395952db9da'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:17.895137 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1364/2771] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:05:17 2022] DELETE /identity/v3/users/844dbb794c95487cb2635395952db9da => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:17.900276 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7284a5-354d-4cbc-9c1d-06ecb8900586 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:17.900732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7284a5-354d-4cbc-9c1d-06ecb8900586 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:17.900847 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7284a5-354d-4cbc-9c1d-06ecb8900586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:17.920094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-0e7284a5-354d-4cbc-9c1d-06ecb8900586 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:17.921274 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0e7284a5-354d-4cbc-9c1d-06ecb8900586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:17.959767 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:17.978673 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1408/2772] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:17.988276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638ebf2-40e4-4636-b090-e7eefc9f26c0 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:17.989704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8638ebf2-40e4-4636-b090-e7eefc9f26c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:18.002947 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:18.048478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8638ebf2-40e4-4636-b090-e7eefc9f26c0 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:18.052160 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad582e93-93a0-4b2c-903a-afdf79607f28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:18.052803 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad582e93-93a0-4b2c-903a-afdf79607f28 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:18.053088 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad582e93-93a0-4b2c-903a-afdf79607f28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:18.053388 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad582e93-93a0-4b2c-903a-afdf79607f28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:18.062318 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad582e93-93a0-4b2c-903a-afdf79607f28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:18.063803 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad582e93-93a0-4b2c-903a-afdf79607f28 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:18.073730 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1365/2773] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:18.338015 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7284a5-354d-4cbc-9c1d-06ecb8900586 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:18.339639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-0e7284a5-354d-4cbc-9c1d-06ecb8900586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:18.352771 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7284a5-354d-4cbc-9c1d-06ecb8900586 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:18.357440 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b645ab-6ccb-49bf-91af-682ac73a4f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:18.358080 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b645ab-6ccb-49bf-91af-682ac73a4f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:18.358359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b645ab-6ccb-49bf-91af-682ac73a4f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:18.358784 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b645ab-6ccb-49bf-91af-682ac73a4f96 admin admin] PATH_INFO: `/v3/projects/999c035701044c69bd451debea36c579` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:18.359973 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b645ab-6ccb-49bf-91af-682ac73a4f96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=999c035701044c69bd451debea36c579)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:18.362618 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b645ab-6ccb-49bf-91af-682ac73a4f96 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:18.412304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-64b645ab-6ccb-49bf-91af-682ac73a4f96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '999c035701044c69bd451debea36c579', 'request_id': 'req-64b645ab-6ccb-49bf-91af-682ac73a4f96'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:18.420118 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-64b645ab-6ccb-49bf-91af-682ac73a4f96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '999c035701044c69bd451debea36c579', 'request_id': 'req-64b645ab-6ccb-49bf-91af-682ac73a4f96'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:18.426290 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-64b645ab-6ccb-49bf-91af-682ac73a4f96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '999c035701044c69bd451debea36c579', 'request_id': 'req-64b645ab-6ccb-49bf-91af-682ac73a4f96'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:18.437061 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-64b645ab-6ccb-49bf-91af-682ac73a4f96 admin admin] The token cache is being invalidate because project 999c035701044c69bd451debea36c579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:05:18.437346 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-64b645ab-6ccb-49bf-91af-682ac73a4f96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:18.437960 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-64b645ab-6ccb-49bf-91af-682ac73a4f96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:18.439489 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1409/2774] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:05:18 2022] DELETE /identity/v3/projects/999c035701044c69bd451debea36c579 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:18.444312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad582e93-93a0-4b2c-903a-afdf79607f28 service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:18.453875 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad582e93-93a0-4b2c-903a-afdf79607f28 service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:18.507687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3cdb2-1b9c-479b-9b46-4c8f864d9994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:18.508312 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3cdb2-1b9c-479b-9b46-4c8f864d9994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:18.508589 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3cdb2-1b9c-479b-9b46-4c8f864d9994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:18.508859 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3cdb2-1b9c-479b-9b46-4c8f864d9994 admin admin] PATH_INFO: `/v3/users/daf2346f3e0049cfb9850a079b33dc6c` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:18.510011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3cdb2-1b9c-479b-9b46-4c8f864d9994 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daf2346f3e0049cfb9850a079b33dc6c)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:18.511921 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3cdb2-1b9c-479b-9b46-4c8f864d9994 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:18.555928 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8cc3cdb2-1b9c-479b-9b46-4c8f864d9994 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daf2346f3e0049cfb9850a079b33dc6c'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:18.567834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8cc3cdb2-1b9c-479b-9b46-4c8f864d9994 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daf2346f3e0049cfb9850a079b33dc6c'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:18.575481 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8cc3cdb2-1b9c-479b-9b46-4c8f864d9994 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daf2346f3e0049cfb9850a079b33dc6c'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:18.576105 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8cc3cdb2-1b9c-479b-9b46-4c8f864d9994 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daf2346f3e0049cfb9850a079b33dc6c'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:18.582879 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8cc3cdb2-1b9c-479b-9b46-4c8f864d9994 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daf2346f3e0049cfb9850a079b33dc6c'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:18.589709 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1366/2775] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:05:18 2022] DELETE /identity/v3/users/daf2346f3e0049cfb9850a079b33dc6c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:18.856433 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b645ab-6ccb-49bf-91af-682ac73a4f96 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:18.858257 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-64b645ab-6ccb-49bf-91af-682ac73a4f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:18.875114 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b645ab-6ccb-49bf-91af-682ac73a4f96 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:18.932019 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679af0e-10bc-428a-be65-4279d5e77c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:18.932649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679af0e-10bc-428a-be65-4279d5e77c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:18.932919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679af0e-10bc-428a-be65-4279d5e77c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:18.933319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679af0e-10bc-428a-be65-4279d5e77c99 admin admin] PATH_INFO: `/v3/projects/16b8432656cb474f807d0ab17341f3f4` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:18.934467 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679af0e-10bc-428a-be65-4279d5e77c99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16b8432656cb474f807d0ab17341f3f4)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:18.936516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679af0e-10bc-428a-be65-4279d5e77c99 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:18.980802 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2679af0e-10bc-428a-be65-4279d5e77c99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16b8432656cb474f807d0ab17341f3f4', 'request_id': 'req-2679af0e-10bc-428a-be65-4279d5e77c99'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:18.987978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2679af0e-10bc-428a-be65-4279d5e77c99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16b8432656cb474f807d0ab17341f3f4', 'request_id': 'req-2679af0e-10bc-428a-be65-4279d5e77c99'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:18.994664 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2679af0e-10bc-428a-be65-4279d5e77c99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16b8432656cb474f807d0ab17341f3f4', 'request_id': 'req-2679af0e-10bc-428a-be65-4279d5e77c99'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:19.000797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:19.006206 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2679af0e-10bc-428a-be65-4279d5e77c99 admin admin] The token cache is being invalidate because project 16b8432656cb474f807d0ab17341f3f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:05:19.006716 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2679af0e-10bc-428a-be65-4279d5e77c99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:19.007852 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-2679af0e-10bc-428a-be65-4279d5e77c99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:19.009689 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1410/2776] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:05:18 2022] DELETE /identity/v3/projects/16b8432656cb474f807d0ab17341f3f4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:19.014652 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f85a3-f926-4cee-bdd2-9278b740cd83 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:19.015154 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f85a3-f926-4cee-bdd2-9278b740cd83 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:19.015264 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58f85a3-f926-4cee-bdd2-9278b740cd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:19.034181 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-e58f85a3-f926-4cee-bdd2-9278b740cd83 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:19.035407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e58f85a3-f926-4cee-bdd2-9278b740cd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:19.065145 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:19.095980 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1367/2777] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 13:05:19.104133 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd30a6d-99fd-4512-9bad-a59e0a8ef921 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:19.104414 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd30a6d-99fd-4512-9bad-a59e0a8ef921 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:19.104717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd30a6d-99fd-4512-9bad-a59e0a8ef921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:19.124561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-edd30a6d-99fd-4512-9bad-a59e0a8ef921 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:19.125748 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-edd30a6d-99fd-4512-9bad-a59e0a8ef921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:19.133523 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1411/2778] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:19.137509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f85a3-f926-4cee-bdd2-9278b740cd83 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:19.147888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58f85a3-f926-4cee-bdd2-9278b740cd83 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:19.152517 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586fd88-da97-41e6-ae53-a60b57d6e07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:19.153117 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586fd88-da97-41e6-ae53-a60b57d6e07c admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:19.153401 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586fd88-da97-41e6-ae53-a60b57d6e07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:19.153676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586fd88-da97-41e6-ae53-a60b57d6e07c admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:19.154373 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586fd88-da97-41e6-ae53-a60b57d6e07c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 13:05:19.154718 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586fd88-da97-41e6-ae53-a60b57d6e07c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:19.156305 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586fd88-da97-41e6-ae53-a60b57d6e07c admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:19.168807 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1368/2779] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 13:05:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 13:05:19.172860 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd30a6d-99fd-4512-9bad-a59e0a8ef921 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:19.181880 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd30a6d-99fd-4512-9bad-a59e0a8ef921 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:19.186439 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d608a8-054e-40d0-8cc6-0654c09086d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:19.187110 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d608a8-054e-40d0-8cc6-0654c09086d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:19.187335 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d608a8-054e-40d0-8cc6-0654c09086d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:19.187606 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d608a8-054e-40d0-8cc6-0654c09086d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:19.188284 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d608a8-054e-40d0-8cc6-0654c09086d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:19.190181 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d608a8-054e-40d0-8cc6-0654c09086d6 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:19.218256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1412/2780] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:05:19 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:19.222186 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586fd88-da97-41e6-ae53-a60b57d6e07c admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:19.232503 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586fd88-da97-41e6-ae53-a60b57d6e07c admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:19.287865 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ac5c0-9a54-4f4d-b4e7-a8f55d60f3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:19.288462 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ac5c0-9a54-4f4d-b4e7-a8f55d60f3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:19.288737 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ac5c0-9a54-4f4d-b4e7-a8f55d60f3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:19.289007 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ac5c0-9a54-4f4d-b4e7-a8f55d60f3a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:19.289665 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ac5c0-9a54-4f4d-b4e7-a8f55d60f3a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:19.291573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ac5c0-9a54-4f4d-b4e7-a8f55d60f3a0 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:19.311651 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1369/2781] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:05:19 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:19.315917 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d608a8-054e-40d0-8cc6-0654c09086d6 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:19.325919 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d608a8-054e-40d0-8cc6-0654c09086d6 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:19.330357 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db0b06-8daf-4f9e-9683-ce3d3349b2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:19.331008 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db0b06-8daf-4f9e-9683-ce3d3349b2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:19.331287 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db0b06-8daf-4f9e-9683-ce3d3349b2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:19.331557 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db0b06-8daf-4f9e-9683-ce3d3349b2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:19.332239 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db0b06-8daf-4f9e-9683-ce3d3349b2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:19.333810 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db0b06-8daf-4f9e-9683-ce3d3349b2e5 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:19.343902 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1413/2782] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:19.347841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ac5c0-9a54-4f4d-b4e7-a8f55d60f3a0 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:19.358337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ac5c0-9a54-4f4d-b4e7-a8f55d60f3a0 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:19.363006 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b650bac7-708e-4d0b-a74d-0fe012a00abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:19.363607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650bac7-708e-4d0b-a74d-0fe012a00abc admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:19.363918 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650bac7-708e-4d0b-a74d-0fe012a00abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:19.364198 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650bac7-708e-4d0b-a74d-0fe012a00abc admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:19.364890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650bac7-708e-4d0b-a74d-0fe012a00abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:19.366475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b650bac7-708e-4d0b-a74d-0fe012a00abc admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:19.377134 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1370/2783] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:19.381255 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db0b06-8daf-4f9e-9683-ce3d3349b2e5 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:19.390951 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db0b06-8daf-4f9e-9683-ce3d3349b2e5 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:19.395453 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef1407-8307-4efb-9495-e016029c0487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:19.396094 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef1407-8307-4efb-9495-e016029c0487 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:19.396370 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef1407-8307-4efb-9495-e016029c0487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:19.396638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef1407-8307-4efb-9495-e016029c0487 admin admin] PATH_INFO: `/v3/projects/51bb304928f840c3b604f59df0af1a88/users/8882fae0ad864bf0a44f5d7ac23ec20b/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:19.411419 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef1407-8307-4efb-9495-e016029c0487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51bb304928f840c3b604f59df0af1a88, user_id=8882fae0ad864bf0a44f5d7ac23ec20b, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:19.414713 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef1407-8307-4efb-9495-e016029c0487 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:19.427658 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1414/2784] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:05:19 2022] PUT /identity/v3/projects/51bb304928f840c3b604f59df0af1a88/users/8882fae0ad864bf0a44f5d7ac23ec20b/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:19.438788 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041f6cc-22fc-4085-93e0-88b9bd435435 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:19.439019 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041f6cc-22fc-4085-93e0-88b9bd435435 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:19.439291 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041f6cc-22fc-4085-93e0-88b9bd435435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:19.467820 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-0041f6cc-22fc-4085-93e0-88b9bd435435 None None] MFA Rules not processed for user `8882fae0ad864bf0a44f5d7ac23ec20b`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:19.469051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-0041f6cc-22fc-4085-93e0-88b9bd435435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:19.533683 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1371/2785] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:19.538084 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef1407-8307-4efb-9495-e016029c0487 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:19.599462 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef1407-8307-4efb-9495-e016029c0487 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:19.604663 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f8e39-3868-4ddd-a273-2a2af12aadb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:19.605271 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f8e39-3868-4ddd-a273-2a2af12aadb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:19.605554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f8e39-3868-4ddd-a273-2a2af12aadb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:19.605828 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f8e39-3868-4ddd-a273-2a2af12aadb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:19.606522 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f8e39-3868-4ddd-a273-2a2af12aadb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:19.608412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646f8e39-3868-4ddd-a273-2a2af12aadb1 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:19.634402 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1415/2786] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:05:19 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 13:05:19.638354 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041f6cc-22fc-4085-93e0-88b9bd435435 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:19.649437 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041f6cc-22fc-4085-93e0-88b9bd435435 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:19.675179 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:19.706298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc536b63-eae4-4015-87ad-a51f67f2737f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:19.706868 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc536b63-eae4-4015-87ad-a51f67f2737f admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:19.707146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc536b63-eae4-4015-87ad-a51f67f2737f admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:19.707417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc536b63-eae4-4015-87ad-a51f67f2737f admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:19.708101 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc536b63-eae4-4015-87ad-a51f67f2737f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:19.709984 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc536b63-eae4-4015-87ad-a51f67f2737f admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:19.730640 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1372/2787] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:05:19 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:19.734493 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f8e39-3868-4ddd-a273-2a2af12aadb1 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:19.745417 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f8e39-3868-4ddd-a273-2a2af12aadb1 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:19.750287 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c9ea4-b36c-4d22-b5a0-9577a4085c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:19.750927 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c9ea4-b36c-4d22-b5a0-9577a4085c71 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:19.751204 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c9ea4-b36c-4d22-b5a0-9577a4085c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:19.751469 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c9ea4-b36c-4d22-b5a0-9577a4085c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:19.752183 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c9ea4-b36c-4d22-b5a0-9577a4085c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:19.753754 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c9ea4-b36c-4d22-b5a0-9577a4085c71 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:19.764359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1416/2788] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:19.768328 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc536b63-eae4-4015-87ad-a51f67f2737f admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:19.778115 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc536b63-eae4-4015-87ad-a51f67f2737f admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:19.782750 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6224ef-1284-4d5d-8d63-5e5d8fa5f6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:19.783407 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6224ef-1284-4d5d-8d63-5e5d8fa5f6b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:19.783695 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6224ef-1284-4d5d-8d63-5e5d8fa5f6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:19.783973 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6224ef-1284-4d5d-8d63-5e5d8fa5f6b1 admin admin] PATH_INFO: `/v3/projects/da645df89bea4ec09f5ef0dc5236f838/users/4be5c9dde3d24134a451ffd34f417658/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:19.799040 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6224ef-1284-4d5d-8d63-5e5d8fa5f6b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da645df89bea4ec09f5ef0dc5236f838, user_id=4be5c9dde3d24134a451ffd34f417658, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:19.801668 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6224ef-1284-4d5d-8d63-5e5d8fa5f6b1 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:19.814756 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1373/2789] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:05:19 2022] PUT /identity/v3/projects/da645df89bea4ec09f5ef0dc5236f838/users/4be5c9dde3d24134a451ffd34f417658/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:19.818588 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c9ea4-b36c-4d22-b5a0-9577a4085c71 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:19.828516 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c9ea4-b36c-4d22-b5a0-9577a4085c71 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:19.884200 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864c37b1-9916-4c46-b571-42daab8313ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:19.884813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c37b1-9916-4c46-b571-42daab8313ff admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:19.885093 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c37b1-9916-4c46-b571-42daab8313ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:19.885365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c37b1-9916-4c46-b571-42daab8313ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:19.886050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c37b1-9916-4c46-b571-42daab8313ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:19.887676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864c37b1-9916-4c46-b571-42daab8313ff admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:19.897766 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1417/2790] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:19.902016 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6224ef-1284-4d5d-8d63-5e5d8fa5f6b1 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:19.912218 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6224ef-1284-4d5d-8d63-5e5d8fa5f6b1 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:19.917639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d26bd-8d54-4c2b-8fb0-7baa7f43cf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:19.918330 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d26bd-8d54-4c2b-8fb0-7baa7f43cf95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:19.918652 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d26bd-8d54-4c2b-8fb0-7baa7f43cf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:19.918928 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d26bd-8d54-4c2b-8fb0-7baa7f43cf95 admin admin] PATH_INFO: `/v3/projects/da645df89bea4ec09f5ef0dc5236f838/users/4be5c9dde3d24134a451ffd34f417658/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:19.920893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d26bd-8d54-4c2b-8fb0-7baa7f43cf95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da645df89bea4ec09f5ef0dc5236f838, user_id=4be5c9dde3d24134a451ffd34f417658, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:19.924089 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d26bd-8d54-4c2b-8fb0-7baa7f43cf95 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:19.933936 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1374/2791] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:05:19 2022] PUT /identity/v3/projects/da645df89bea4ec09f5ef0dc5236f838/users/4be5c9dde3d24134a451ffd34f417658/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:19.944098 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be88bb-43f9-4900-a99d-968881fbf82a None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:19.944580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be88bb-43f9-4900-a99d-968881fbf82a None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:19.944739 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be88bb-43f9-4900-a99d-968881fbf82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:19.952731 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:19.973571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-14be88bb-43f9-4900-a99d-968881fbf82a None None] MFA Rules not processed for user `4be5c9dde3d24134a451ffd34f417658`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:19.974804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-14be88bb-43f9-4900-a99d-968881fbf82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:20.041898 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1418/2792] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:20.053930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d26bd-8d54-4c2b-8fb0-7baa7f43cf95 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:20.055700 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-711d26bd-8d54-4c2b-8fb0-7baa7f43cf95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:20.112337 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d26bd-8d54-4c2b-8fb0-7baa7f43cf95 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:20.116489 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4f92e-3080-40f2-9571-e3399d48fc78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:20.117098 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4f92e-3080-40f2-9571-e3399d48fc78 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:20.117378 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4f92e-3080-40f2-9571-e3399d48fc78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:20.117652 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4f92e-3080-40f2-9571-e3399d48fc78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:20.126601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4f92e-3080-40f2-9571-e3399d48fc78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:20.128050 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4f92e-3080-40f2-9571-e3399d48fc78 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:20.138059 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1375/2793] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:20.181950 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be88bb-43f9-4900-a99d-968881fbf82a None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:20.183418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-14be88bb-43f9-4900-a99d-968881fbf82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:20.196068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be88bb-43f9-4900-a99d-968881fbf82a None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:20.199706 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0973f6c2-10b0-4fab-b676-b8e933670bfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:20.200315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973f6c2-10b0-4fab-b676-b8e933670bfc service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:20.200593 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973f6c2-10b0-4fab-b676-b8e933670bfc service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:20.200866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0973f6c2-10b0-4fab-b676-b8e933670bfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:20.256145 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973f6c2-10b0-4fab-b676-b8e933670bfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:20.258090 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0973f6c2-10b0-4fab-b676-b8e933670bfc service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:20.267127 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1419/2794] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:23.523065 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4f92e-3080-40f2-9571-e3399d48fc78 service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:23.583869 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4f92e-3080-40f2-9571-e3399d48fc78 service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:23.588496 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b3017-ddc6-4b88-abd1-8ddfb730b0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:23.589185 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b3017-ddc6-4b88-abd1-8ddfb730b0fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:23.589469 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b3017-ddc6-4b88-abd1-8ddfb730b0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:23.589746 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b3017-ddc6-4b88-abd1-8ddfb730b0fe admin admin] PATH_INFO: `/v3/users/8882fae0ad864bf0a44f5d7ac23ec20b` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:23.590943 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b3017-ddc6-4b88-abd1-8ddfb730b0fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=8882fae0ad864bf0a44f5d7ac23ec20b)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:23.592880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b3017-ddc6-4b88-abd1-8ddfb730b0fe admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:23.620192 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:23.638574 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c76b3017-ddc6-4b88-abd1-8ddfb730b0fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8882fae0ad864bf0a44f5d7ac23ec20b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:23.650935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c76b3017-ddc6-4b88-abd1-8ddfb730b0fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8882fae0ad864bf0a44f5d7ac23ec20b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:23.658730 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c76b3017-ddc6-4b88-abd1-8ddfb730b0fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8882fae0ad864bf0a44f5d7ac23ec20b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:23.659341 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c76b3017-ddc6-4b88-abd1-8ddfb730b0fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8882fae0ad864bf0a44f5d7ac23ec20b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:23.666735 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c76b3017-ddc6-4b88-abd1-8ddfb730b0fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8882fae0ad864bf0a44f5d7ac23ec20b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:23.672746 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1376/2795] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:05:23 2022] DELETE /identity/v3/users/8882fae0ad864bf0a44f5d7ac23ec20b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:23.677187 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0ad57-d2aa-48c1-a9f0-8b6287777db2 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:23.677664 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0ad57-d2aa-48c1-a9f0-8b6287777db2 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:23.677779 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c0ad57-d2aa-48c1-a9f0-8b6287777db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:23.696906 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-59c0ad57-d2aa-48c1-a9f0-8b6287777db2 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:23.698616 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-59c0ad57-d2aa-48c1-a9f0-8b6287777db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:23.757140 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1420/2796] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:23.766811 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b3017-ddc6-4b88-abd1-8ddfb730b0fe admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:23.768225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-c76b3017-ddc6-4b88-abd1-8ddfb730b0fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:23.823928 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b3017-ddc6-4b88-abd1-8ddfb730b0fe admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:23.827548 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ae769-efaa-47a2-82ab-d66f69fb6f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:23.828144 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ae769-efaa-47a2-82ab-d66f69fb6f87 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:23.828416 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ae769-efaa-47a2-82ab-d66f69fb6f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:23.828682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ae769-efaa-47a2-82ab-d66f69fb6f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:23.837980 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ae769-efaa-47a2-82ab-d66f69fb6f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:23.839417 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ae769-efaa-47a2-82ab-d66f69fb6f87 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:23.848588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1377/2797] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:24.113326 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c0ad57-d2aa-48c1-a9f0-8b6287777db2 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:24.114827 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-59c0ad57-d2aa-48c1-a9f0-8b6287777db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:24.126921 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c0ad57-d2aa-48c1-a9f0-8b6287777db2 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:24.131614 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f0f25-6560-4f95-9e8a-cff939f1d722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:24.132774 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f0f25-6560-4f95-9e8a-cff939f1d722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:24.133056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f0f25-6560-4f95-9e8a-cff939f1d722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:24.133365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f0f25-6560-4f95-9e8a-cff939f1d722 admin admin] PATH_INFO: `/v3/projects/51bb304928f840c3b604f59df0af1a88` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:24.134560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f0f25-6560-4f95-9e8a-cff939f1d722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51bb304928f840c3b604f59df0af1a88)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:24.136466 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f0f25-6560-4f95-9e8a-cff939f1d722 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:24.161561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:24.183781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-7f0f0f25-6560-4f95-9e8a-cff939f1d722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51bb304928f840c3b604f59df0af1a88', 'request_id': 'req-7f0f0f25-6560-4f95-9e8a-cff939f1d722'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:24.191011 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-7f0f0f25-6560-4f95-9e8a-cff939f1d722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51bb304928f840c3b604f59df0af1a88', 'request_id': 'req-7f0f0f25-6560-4f95-9e8a-cff939f1d722'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:24.197794 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-7f0f0f25-6560-4f95-9e8a-cff939f1d722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51bb304928f840c3b604f59df0af1a88', 'request_id': 'req-7f0f0f25-6560-4f95-9e8a-cff939f1d722'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:24.207873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-7f0f0f25-6560-4f95-9e8a-cff939f1d722 admin admin] The token cache is being invalidate because project 51bb304928f840c3b604f59df0af1a88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:05:24.208160 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-7f0f0f25-6560-4f95-9e8a-cff939f1d722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:24.208807 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-7f0f0f25-6560-4f95-9e8a-cff939f1d722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:24.210978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1421/2798] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:05:24 2022] DELETE /identity/v3/projects/51bb304928f840c3b604f59df0af1a88 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 13:05:24.215097 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ae769-efaa-47a2-82ab-d66f69fb6f87 service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:24.224929 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ae769-efaa-47a2-82ab-d66f69fb6f87 service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:24.279440 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6fd92-bf62-45eb-bbfe-9367f189f1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:24.280074 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6fd92-bf62-45eb-bbfe-9367f189f1d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:24.280359 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6fd92-bf62-45eb-bbfe-9367f189f1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:24.280637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6fd92-bf62-45eb-bbfe-9367f189f1d6 admin admin] PATH_INFO: `/v3/users/4be5c9dde3d24134a451ffd34f417658` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:24.281803 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6fd92-bf62-45eb-bbfe-9367f189f1d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4be5c9dde3d24134a451ffd34f417658)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:24.283753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b6fd92-bf62-45eb-bbfe-9367f189f1d6 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:24.328951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c0b6fd92-bf62-45eb-bbfe-9367f189f1d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4be5c9dde3d24134a451ffd34f417658'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:24.330107 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:24.342626 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c0b6fd92-bf62-45eb-bbfe-9367f189f1d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4be5c9dde3d24134a451ffd34f417658'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:24.349869 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c0b6fd92-bf62-45eb-bbfe-9367f189f1d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4be5c9dde3d24134a451ffd34f417658'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:24.350507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c0b6fd92-bf62-45eb-bbfe-9367f189f1d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4be5c9dde3d24134a451ffd34f417658'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:24.358062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c0b6fd92-bf62-45eb-bbfe-9367f189f1d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4be5c9dde3d24134a451ffd34f417658'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:24.363800 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1378/2799] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:05:24 2022] DELETE /identity/v3/users/4be5c9dde3d24134a451ffd34f417658 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:24.630878 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f0f25-6560-4f95-9e8a-cff939f1d722 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:24.632395 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-7f0f0f25-6560-4f95-9e8a-cff939f1d722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:24.652531 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f0f25-6560-4f95-9e8a-cff939f1d722 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:24.709137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5df6e2-4c90-4b8d-b811-47cbdb70edcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:24.709781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5df6e2-4c90-4b8d-b811-47cbdb70edcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:24.710607 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5df6e2-4c90-4b8d-b811-47cbdb70edcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:24.710885 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5df6e2-4c90-4b8d-b811-47cbdb70edcd admin admin] PATH_INFO: `/v3/projects/da645df89bea4ec09f5ef0dc5236f838` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:24.712045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5df6e2-4c90-4b8d-b811-47cbdb70edcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=da645df89bea4ec09f5ef0dc5236f838)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:24.713946 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5df6e2-4c90-4b8d-b811-47cbdb70edcd admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:24.759593 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fb5df6e2-4c90-4b8d-b811-47cbdb70edcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da645df89bea4ec09f5ef0dc5236f838', 'request_id': 'req-fb5df6e2-4c90-4b8d-b811-47cbdb70edcd'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:24.766308 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fb5df6e2-4c90-4b8d-b811-47cbdb70edcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da645df89bea4ec09f5ef0dc5236f838', 'request_id': 'req-fb5df6e2-4c90-4b8d-b811-47cbdb70edcd'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:24.772314 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fb5df6e2-4c90-4b8d-b811-47cbdb70edcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da645df89bea4ec09f5ef0dc5236f838', 'request_id': 'req-fb5df6e2-4c90-4b8d-b811-47cbdb70edcd'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:24.782936 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fb5df6e2-4c90-4b8d-b811-47cbdb70edcd admin admin] The token cache is being invalidate because project da645df89bea4ec09f5ef0dc5236f838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:05:24.783225 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fb5df6e2-4c90-4b8d-b811-47cbdb70edcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:24.783845 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-fb5df6e2-4c90-4b8d-b811-47cbdb70edcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:24.785345 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1422/2800] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:05:24 2022] DELETE /identity/v3/projects/da645df89bea4ec09f5ef0dc5236f838 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:24.790359 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b139843-8953-4deb-82bc-81a222faaa47 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:24.790873 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b139843-8953-4deb-82bc-81a222faaa47 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:24.790982 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b139843-8953-4deb-82bc-81a222faaa47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:24.810342 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-9b139843-8953-4deb-82bc-81a222faaa47 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:24.811586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-9b139843-8953-4deb-82bc-81a222faaa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:24.870111 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1379/2801] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:24.877944 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dd1b4-92ba-4bf6-b37a-0717d672c335 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:24.878229 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dd1b4-92ba-4bf6-b37a-0717d672c335 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:24.878524 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dd1b4-92ba-4bf6-b37a-0717d672c335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:24.898841 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-f70dd1b4-92ba-4bf6-b37a-0717d672c335 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:24.900035 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f70dd1b4-92ba-4bf6-b37a-0717d672c335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:24.907867 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1423/2802] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:24.912070 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b139843-8953-4deb-82bc-81a222faaa47 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:24.922030 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b139843-8953-4deb-82bc-81a222faaa47 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:24.926749 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4b176-38f9-4325-8ae4-caad9a2fe90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:24.927356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4b176-38f9-4325-8ae4-caad9a2fe90c admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:24.927638 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4b176-38f9-4325-8ae4-caad9a2fe90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:24.927911 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4b176-38f9-4325-8ae4-caad9a2fe90c admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:24.928602 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b4b176-38f9-4325-8ae4-caad9a2fe90c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 13:05:24.928905 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b4b176-38f9-4325-8ae4-caad9a2fe90c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:24.930498 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b4b176-38f9-4325-8ae4-caad9a2fe90c admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:24.942572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1380/2803] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 13:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 13:05:24.947449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70dd1b4-92ba-4bf6-b37a-0717d672c335 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:24.956575 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70dd1b4-92ba-4bf6-b37a-0717d672c335 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:24.962037 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076f057-a59d-4560-b76f-8ecaa5f2ff9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:24.962724 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076f057-a59d-4560-b76f-8ecaa5f2ff9e admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:24.963011 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076f057-a59d-4560-b76f-8ecaa5f2ff9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:24.963285 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f076f057-a59d-4560-b76f-8ecaa5f2ff9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:24.963947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f076f057-a59d-4560-b76f-8ecaa5f2ff9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:24.965828 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f076f057-a59d-4560-b76f-8ecaa5f2ff9e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:24.992180 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1424/2804] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:05:24 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:24.996524 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4b176-38f9-4325-8ae4-caad9a2fe90c admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:25.006381 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4b176-38f9-4325-8ae4-caad9a2fe90c admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:25.060327 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81662331-5f40-426a-a1ea-9d18dea48b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:25.060931 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81662331-5f40-426a-a1ea-9d18dea48b17 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:25.061207 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81662331-5f40-426a-a1ea-9d18dea48b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:25.061520 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81662331-5f40-426a-a1ea-9d18dea48b17 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:25.062319 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81662331-5f40-426a-a1ea-9d18dea48b17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:25.064232 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81662331-5f40-426a-a1ea-9d18dea48b17 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:25.084853 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1381/2805] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:05:24 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:25.089608 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076f057-a59d-4560-b76f-8ecaa5f2ff9e admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:25.099803 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f076f057-a59d-4560-b76f-8ecaa5f2ff9e admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:25.104388 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bd813-0f69-4a01-8215-8fcba2dce7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:25.104989 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bd813-0f69-4a01-8215-8fcba2dce7aa admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:25.105265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bd813-0f69-4a01-8215-8fcba2dce7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:25.105535 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617bd813-0f69-4a01-8215-8fcba2dce7aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:25.106216 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617bd813-0f69-4a01-8215-8fcba2dce7aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:25.107831 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617bd813-0f69-4a01-8215-8fcba2dce7aa admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:25.118243 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1425/2806] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:25.121881 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81662331-5f40-426a-a1ea-9d18dea48b17 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:25.131715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81662331-5f40-426a-a1ea-9d18dea48b17 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:25.136849 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de3869-2893-4b56-a2b4-fed6c41e633c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:25.137441 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de3869-2893-4b56-a2b4-fed6c41e633c admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:25.137718 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de3869-2893-4b56-a2b4-fed6c41e633c admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:25.137988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de3869-2893-4b56-a2b4-fed6c41e633c admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:25.138706 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de3869-2893-4b56-a2b4-fed6c41e633c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:25.140274 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de3869-2893-4b56-a2b4-fed6c41e633c admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:25.150466 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1382/2807] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:25.154580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bd813-0f69-4a01-8215-8fcba2dce7aa admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:25.159889 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:25.165985 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617bd813-0f69-4a01-8215-8fcba2dce7aa admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:25.171998 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddee4a6-1f7d-4d73-8eee-e1401c21ae24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:25.172681 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddee4a6-1f7d-4d73-8eee-e1401c21ae24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:25.172957 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddee4a6-1f7d-4d73-8eee-e1401c21ae24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:25.173227 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddee4a6-1f7d-4d73-8eee-e1401c21ae24 admin admin] PATH_INFO: `/v3/projects/6394b8f13e81438793b2977d0d7141e7/users/1a5502f71b0e40b9a53031f4f0ba1877/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:25.188141 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddee4a6-1f7d-4d73-8eee-e1401c21ae24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6394b8f13e81438793b2977d0d7141e7, user_id=1a5502f71b0e40b9a53031f4f0ba1877, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:25.190777 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddee4a6-1f7d-4d73-8eee-e1401c21ae24 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:25.203661 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1426/2808] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:05:25 2022] PUT /identity/v3/projects/6394b8f13e81438793b2977d0d7141e7/users/1a5502f71b0e40b9a53031f4f0ba1877/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:25.215047 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bc6be-2e8d-4797-8d58-fc27d9dda6e4 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:25.215266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bc6be-2e8d-4797-8d58-fc27d9dda6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:25.215543 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1bc6be-2e8d-4797-8d58-fc27d9dda6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:25.242291 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-4c1bc6be-2e8d-4797-8d58-fc27d9dda6e4 None None] MFA Rules not processed for user `1a5502f71b0e40b9a53031f4f0ba1877`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:25.243514 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-4c1bc6be-2e8d-4797-8d58-fc27d9dda6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:25.261841 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:25.308627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1383/2809] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:25.313055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddee4a6-1f7d-4d73-8eee-e1401c21ae24 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:25.373537 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddee4a6-1f7d-4d73-8eee-e1401c21ae24 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:25.378107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a0fef-e7a8-4067-9871-1e54e4e27884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:25.378729 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a0fef-e7a8-4067-9871-1e54e4e27884 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:25.379007 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a0fef-e7a8-4067-9871-1e54e4e27884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:25.379275 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a0fef-e7a8-4067-9871-1e54e4e27884 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:25.379928 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a0fef-e7a8-4067-9871-1e54e4e27884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:25.382426 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a0fef-e7a8-4067-9871-1e54e4e27884 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:25.407714 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1427/2810] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:05:25 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:25.411780 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bc6be-2e8d-4797-8d58-fc27d9dda6e4 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:25.422810 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1bc6be-2e8d-4797-8d58-fc27d9dda6e4 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:25.478397 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60080d7f-78a3-4348-acb0-da4358067fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:25.478976 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60080d7f-78a3-4348-acb0-da4358067fef admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:25.479364 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60080d7f-78a3-4348-acb0-da4358067fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:25.479677 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60080d7f-78a3-4348-acb0-da4358067fef admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:25.480352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60080d7f-78a3-4348-acb0-da4358067fef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:25.482257 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60080d7f-78a3-4348-acb0-da4358067fef admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:25.503440 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1384/2811] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:05:25 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:25.507459 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a0fef-e7a8-4067-9871-1e54e4e27884 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:25.517742 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a0fef-e7a8-4067-9871-1e54e4e27884 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:25.522383 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e9131-38a7-4be2-917d-01a5fe2a6747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:25.523014 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e9131-38a7-4be2-917d-01a5fe2a6747 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:25.523297 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e9131-38a7-4be2-917d-01a5fe2a6747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:25.523572 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e9131-38a7-4be2-917d-01a5fe2a6747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:25.524264 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e9131-38a7-4be2-917d-01a5fe2a6747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:25.525844 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e9131-38a7-4be2-917d-01a5fe2a6747 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:25.537298 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1428/2812] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:25.541652 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60080d7f-78a3-4348-acb0-da4358067fef admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:25.551803 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60080d7f-78a3-4348-acb0-da4358067fef admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:25.557120 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e6e324-8259-4fd7-bf03-d78783fcc02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:25.557765 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6e324-8259-4fd7-bf03-d78783fcc02b admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:25.558069 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6e324-8259-4fd7-bf03-d78783fcc02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:25.558509 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6e324-8259-4fd7-bf03-d78783fcc02b admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:25.559389 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e6e324-8259-4fd7-bf03-d78783fcc02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:25.560970 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e6e324-8259-4fd7-bf03-d78783fcc02b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:25.570794 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1385/2813] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:25.574980 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e9131-38a7-4be2-917d-01a5fe2a6747 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:25.584652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e9131-38a7-4be2-917d-01a5fe2a6747 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:25.589521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d498a-5dd7-43e6-a0bc-6e45df00b275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:25.590170 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d498a-5dd7-43e6-a0bc-6e45df00b275 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:25.590452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d498a-5dd7-43e6-a0bc-6e45df00b275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:25.590760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d498a-5dd7-43e6-a0bc-6e45df00b275 admin admin] PATH_INFO: `/v3/projects/d39b3a1fccad46f9b363fdfdbf642573/users/71fb908e8ccc471d81a8fee7853f4dc4/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:25.605481 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d498a-5dd7-43e6-a0bc-6e45df00b275 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d39b3a1fccad46f9b363fdfdbf642573, user_id=71fb908e8ccc471d81a8fee7853f4dc4, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:25.608158 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741d498a-5dd7-43e6-a0bc-6e45df00b275 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:25.621442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1429/2814] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:05:25 2022] PUT /identity/v3/projects/d39b3a1fccad46f9b363fdfdbf642573/users/71fb908e8ccc471d81a8fee7853f4dc4/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:25.631327 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf5aa1-893f-42f0-b352-6368a8eb09c9 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:25.631666 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf5aa1-893f-42f0-b352-6368a8eb09c9 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:25.631977 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf5aa1-893f-42f0-b352-6368a8eb09c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:25.658959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-aacf5aa1-893f-42f0-b352-6368a8eb09c9 None None] MFA Rules not processed for user `71fb908e8ccc471d81a8fee7853f4dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:25.660445 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-aacf5aa1-893f-42f0-b352-6368a8eb09c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:25.723788 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1386/2815] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:25.728736 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d498a-5dd7-43e6-a0bc-6e45df00b275 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:25.787781 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d498a-5dd7-43e6-a0bc-6e45df00b275 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:25.793453 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93d14b-325d-4929-8edd-2c00097ef71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:25.794062 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93d14b-325d-4929-8edd-2c00097ef71f admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:25.794365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93d14b-325d-4929-8edd-2c00097ef71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:25.794665 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93d14b-325d-4929-8edd-2c00097ef71f admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:25.795336 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93d14b-325d-4929-8edd-2c00097ef71f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:25.797203 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93d14b-325d-4929-8edd-2c00097ef71f admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:25.823898 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1430/2816] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:05:25 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:25.827906 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf5aa1-893f-42f0-b352-6368a8eb09c9 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:25.838518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf5aa1-893f-42f0-b352-6368a8eb09c9 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:25.864011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:25.895188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ee5e5-fd08-4750-8786-b11f274d33fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:25.895813 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ee5e5-fd08-4750-8786-b11f274d33fb admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:25.896113 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ee5e5-fd08-4750-8786-b11f274d33fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:25.896388 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270ee5e5-fd08-4750-8786-b11f274d33fb admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:25.897051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ee5e5-fd08-4750-8786-b11f274d33fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:25.899564 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270ee5e5-fd08-4750-8786-b11f274d33fb admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:25.920678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1387/2817] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:05:25 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:25.924910 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93d14b-325d-4929-8edd-2c00097ef71f admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:25.935069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93d14b-325d-4929-8edd-2c00097ef71f admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:25.939613 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894ec55-e37d-41f1-836b-f119d988f35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:25.940215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894ec55-e37d-41f1-836b-f119d988f35a admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:25.940495 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894ec55-e37d-41f1-836b-f119d988f35a admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:25.940763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894ec55-e37d-41f1-836b-f119d988f35a admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:25.941448 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894ec55-e37d-41f1-836b-f119d988f35a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:25.943051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894ec55-e37d-41f1-836b-f119d988f35a admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:25.953329 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1431/2818] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:25.957314 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ee5e5-fd08-4750-8786-b11f274d33fb admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:25.968377 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270ee5e5-fd08-4750-8786-b11f274d33fb admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:25.973017 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c4f1d6-3ae5-4292-93b1-81cdbdca9619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:25.973669 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4f1d6-3ae5-4292-93b1-81cdbdca9619 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:25.973948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4f1d6-3ae5-4292-93b1-81cdbdca9619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:25.974251 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c4f1d6-3ae5-4292-93b1-81cdbdca9619 admin admin] PATH_INFO: `/v3/projects/216c2acf71d24224af45dce343c1429d/users/a4e0b41d952c476abc419ee384fb51dc/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:25.989141 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c4f1d6-3ae5-4292-93b1-81cdbdca9619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=216c2acf71d24224af45dce343c1429d, user_id=a4e0b41d952c476abc419ee384fb51dc, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:25.991793 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c4f1d6-3ae5-4292-93b1-81cdbdca9619 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:26.004531 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1388/2819] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:05:25 2022] PUT /identity/v3/projects/216c2acf71d24224af45dce343c1429d/users/a4e0b41d952c476abc419ee384fb51dc/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:26.008368 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894ec55-e37d-41f1-836b-f119d988f35a admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:26.013523 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:26.019873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894ec55-e37d-41f1-836b-f119d988f35a admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:26.075277 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f8725-306d-4885-ac61-dce53d475b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:26.075887 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f8725-306d-4885-ac61-dce53d475b52 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:26.076169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f8725-306d-4885-ac61-dce53d475b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:26.076442 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f8725-306d-4885-ac61-dce53d475b52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:26.077125 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f8725-306d-4885-ac61-dce53d475b52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:26.078755 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f8725-306d-4885-ac61-dce53d475b52 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:26.088717 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1432/2820] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:26.092767 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c4f1d6-3ae5-4292-93b1-81cdbdca9619 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:26.103390 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c4f1d6-3ae5-4292-93b1-81cdbdca9619 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:26.108010 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfd7fa-6e14-422a-a303-32cdd3a2c403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:26.108665 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfd7fa-6e14-422a-a303-32cdd3a2c403 admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:26.108942 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfd7fa-6e14-422a-a303-32cdd3a2c403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:26.109215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfd7fa-6e14-422a-a303-32cdd3a2c403 admin admin] PATH_INFO: `/v3/projects/216c2acf71d24224af45dce343c1429d/users/a4e0b41d952c476abc419ee384fb51dc/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:26.111188 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cfd7fa-6e14-422a-a303-32cdd3a2c403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=216c2acf71d24224af45dce343c1429d, user_id=a4e0b41d952c476abc419ee384fb51dc, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:26.113777 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cfd7fa-6e14-422a-a303-32cdd3a2c403 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:26.123892 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1389/2821] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:05:26 2022] PUT /identity/v3/projects/216c2acf71d24224af45dce343c1429d/users/a4e0b41d952c476abc419ee384fb51dc/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:26.134975 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f64310-4c4e-4152-a3fb-6cc0487f1d84 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:26.135468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f64310-4c4e-4152-a3fb-6cc0487f1d84 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:26.135757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f64310-4c4e-4152-a3fb-6cc0487f1d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:26.163023 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-a4f64310-4c4e-4152-a3fb-6cc0487f1d84 None None] MFA Rules not processed for user `a4e0b41d952c476abc419ee384fb51dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:26.164849 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a4f64310-4c4e-4152-a3fb-6cc0487f1d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:26.231493 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1433/2822] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:26.242055 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfd7fa-6e14-422a-a303-32cdd3a2c403 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:26.243751 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e5cfd7fa-6e14-422a-a303-32cdd3a2c403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:26.301966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfd7fa-6e14-422a-a303-32cdd3a2c403 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:26.305583 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe03a32-3c78-42e4-929b-00be666d0e8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:26.306187 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe03a32-3c78-42e4-929b-00be666d0e8d service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:26.306462 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe03a32-3c78-42e4-929b-00be666d0e8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:26.306764 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe03a32-3c78-42e4-929b-00be666d0e8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:26.315653 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe03a32-3c78-42e4-929b-00be666d0e8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:26.317099 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe03a32-3c78-42e4-929b-00be666d0e8d service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:26.327151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1390/2823] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:26.812617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f64310-4c4e-4152-a3fb-6cc0487f1d84 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:26.814045 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-a4f64310-4c4e-4152-a3fb-6cc0487f1d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:26.826626 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f64310-4c4e-4152-a3fb-6cc0487f1d84 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:26.830156 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35404b97-bb37-4222-a918-eef41f8799eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:26.830813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35404b97-bb37-4222-a918-eef41f8799eb service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:26.831091 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35404b97-bb37-4222-a918-eef41f8799eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:26.831366 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35404b97-bb37-4222-a918-eef41f8799eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:26.888449 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35404b97-bb37-4222-a918-eef41f8799eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:26.889891 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35404b97-bb37-4222-a918-eef41f8799eb service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:26.898837 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1434/2824] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:30.883518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe03a32-3c78-42e4-929b-00be666d0e8d service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:30.892152 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe03a32-3c78-42e4-929b-00be666d0e8d service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:30.895760 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4e59b-2177-45be-aaeb-1780dfbd3909 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:30.896356 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4e59b-2177-45be-aaeb-1780dfbd3909 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:30.896637 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4e59b-2177-45be-aaeb-1780dfbd3909 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:30.896905 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4e59b-2177-45be-aaeb-1780dfbd3909 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:30.952344 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4e59b-2177-45be-aaeb-1780dfbd3909 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:30.953685 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4e59b-2177-45be-aaeb-1780dfbd3909 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:30.962476 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1391/2825] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:33.002926 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35404b97-bb37-4222-a918-eef41f8799eb service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:33.028750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=100 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:33.065858 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35404b97-bb37-4222-a918-eef41f8799eb service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:33.070872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e6cfc-fd80-43f2-b5b2-34dc6acd6392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:33.071503 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e6cfc-fd80-43f2-b5b2-34dc6acd6392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:33.071783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e6cfc-fd80-43f2-b5b2-34dc6acd6392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:33.072055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e6cfc-fd80-43f2-b5b2-34dc6acd6392 admin admin] PATH_INFO: `/v3/users/1a5502f71b0e40b9a53031f4f0ba1877` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:33.073204 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e6cfc-fd80-43f2-b5b2-34dc6acd6392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a5502f71b0e40b9a53031f4f0ba1877)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:33.075169 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e6cfc-fd80-43f2-b5b2-34dc6acd6392 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:33.121674 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-591e6cfc-fd80-43f2-b5b2-34dc6acd6392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a5502f71b0e40b9a53031f4f0ba1877'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:33.128281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-591e6cfc-fd80-43f2-b5b2-34dc6acd6392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a5502f71b0e40b9a53031f4f0ba1877'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:33.140264 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-591e6cfc-fd80-43f2-b5b2-34dc6acd6392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a5502f71b0e40b9a53031f4f0ba1877'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:33.140873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-591e6cfc-fd80-43f2-b5b2-34dc6acd6392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a5502f71b0e40b9a53031f4f0ba1877'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:33.141463 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-591e6cfc-fd80-43f2-b5b2-34dc6acd6392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a5502f71b0e40b9a53031f4f0ba1877'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:33.153833 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1435/2826] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:05:33 2022] DELETE /identity/v3/users/1a5502f71b0e40b9a53031f4f0ba1877 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:33.158243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70eadbe-e475-4a8d-afcd-2e413b27156e None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:33.158793 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70eadbe-e475-4a8d-afcd-2e413b27156e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:33.158906 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70eadbe-e475-4a8d-afcd-2e413b27156e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:33.177718 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-b70eadbe-e475-4a8d-afcd-2e413b27156e None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:33.178946 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-b70eadbe-e475-4a8d-afcd-2e413b27156e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:33.195349 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:33.237706 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1392/2827] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:33.247652 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e6cfc-fd80-43f2-b5b2-34dc6acd6392 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:33.249073 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-591e6cfc-fd80-43f2-b5b2-34dc6acd6392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:33.308154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e6cfc-fd80-43f2-b5b2-34dc6acd6392 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:33.311806 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d9bbf-2a43-4e0a-b5f6-9830aa83ccd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:33.312428 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d9bbf-2a43-4e0a-b5f6-9830aa83ccd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:33.312707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d9bbf-2a43-4e0a-b5f6-9830aa83ccd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:33.312978 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d9bbf-2a43-4e0a-b5f6-9830aa83ccd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:33.322570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d9bbf-2a43-4e0a-b5f6-9830aa83ccd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:33.323986 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d9bbf-2a43-4e0a-b5f6-9830aa83ccd2 service neutron] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:33.333682 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1436/2828] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:33.869690 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70eadbe-e475-4a8d-afcd-2e413b27156e None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:33.871306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-b70eadbe-e475-4a8d-afcd-2e413b27156e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:33.885196 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70eadbe-e475-4a8d-afcd-2e413b27156e None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:33.891621 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dde807-6c0f-4731-9397-2e05ec27c1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:33.892300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dde807-6c0f-4731-9397-2e05ec27c1b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:33.892607 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dde807-6c0f-4731-9397-2e05ec27c1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:33.892874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dde807-6c0f-4731-9397-2e05ec27c1b5 admin admin] PATH_INFO: `/v3/projects/6394b8f13e81438793b2977d0d7141e7` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:33.894890 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dde807-6c0f-4731-9397-2e05ec27c1b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6394b8f13e81438793b2977d0d7141e7)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:33.897157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dde807-6c0f-4731-9397-2e05ec27c1b5 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:33.943132 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-94dde807-6c0f-4731-9397-2e05ec27c1b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6394b8f13e81438793b2977d0d7141e7', 'request_id': 'req-94dde807-6c0f-4731-9397-2e05ec27c1b5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:33.950529 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-94dde807-6c0f-4731-9397-2e05ec27c1b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6394b8f13e81438793b2977d0d7141e7', 'request_id': 'req-94dde807-6c0f-4731-9397-2e05ec27c1b5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:33.956661 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-94dde807-6c0f-4731-9397-2e05ec27c1b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6394b8f13e81438793b2977d0d7141e7', 'request_id': 'req-94dde807-6c0f-4731-9397-2e05ec27c1b5'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:33.967986 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-94dde807-6c0f-4731-9397-2e05ec27c1b5 admin admin] The token cache is being invalidate because project 6394b8f13e81438793b2977d0d7141e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:05:33.968267 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-94dde807-6c0f-4731-9397-2e05ec27c1b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:33.968891 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-94dde807-6c0f-4731-9397-2e05ec27c1b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:33.970365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1393/2829] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:05:33 2022] DELETE /identity/v3/projects/6394b8f13e81438793b2977d0d7141e7 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:33.974591 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d9bbf-2a43-4e0a-b5f6-9830aa83ccd2 service neutron] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:33.984321 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d9bbf-2a43-4e0a-b5f6-9830aa83ccd2 service neutron] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:34.025215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:34.041283 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacbd4c-13eb-474d-bb0d-b6a6bf431db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:34.042188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacbd4c-13eb-474d-bb0d-b6a6bf431db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:34.042471 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacbd4c-13eb-474d-bb0d-b6a6bf431db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:34.042770 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacbd4c-13eb-474d-bb0d-b6a6bf431db8 admin admin] PATH_INFO: `/v3/users/71fb908e8ccc471d81a8fee7853f4dc4` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:34.043917 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacbd4c-13eb-474d-bb0d-b6a6bf431db8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71fb908e8ccc471d81a8fee7853f4dc4)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:34.045860 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacbd4c-13eb-474d-bb0d-b6a6bf431db8 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:34.092325 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1eacbd4c-13eb-474d-bb0d-b6a6bf431db8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71fb908e8ccc471d81a8fee7853f4dc4'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:34.099130 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1eacbd4c-13eb-474d-bb0d-b6a6bf431db8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71fb908e8ccc471d81a8fee7853f4dc4'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:34.110896 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1eacbd4c-13eb-474d-bb0d-b6a6bf431db8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71fb908e8ccc471d81a8fee7853f4dc4'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:34.111572 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1eacbd4c-13eb-474d-bb0d-b6a6bf431db8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71fb908e8ccc471d81a8fee7853f4dc4'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:34.112163 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-1eacbd4c-13eb-474d-bb0d-b6a6bf431db8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71fb908e8ccc471d81a8fee7853f4dc4'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:34.131107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1437/2830] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:05:33 2022] DELETE /identity/v3/users/71fb908e8ccc471d81a8fee7853f4dc4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:34.398242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dde807-6c0f-4731-9397-2e05ec27c1b5 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:34.399741 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-94dde807-6c0f-4731-9397-2e05ec27c1b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:34.412641 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dde807-6c0f-4731-9397-2e05ec27c1b5 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:34.469116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76abd9-3d0f-43c8-845d-493168028e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:34.469919 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76abd9-3d0f-43c8-845d-493168028e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:34.470203 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76abd9-3d0f-43c8-845d-493168028e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:34.470495 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76abd9-3d0f-43c8-845d-493168028e47 admin admin] PATH_INFO: `/v3/projects/d39b3a1fccad46f9b363fdfdbf642573` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:34.471704 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76abd9-3d0f-43c8-845d-493168028e47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d39b3a1fccad46f9b363fdfdbf642573)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:34.473639 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76abd9-3d0f-43c8-845d-493168028e47 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:34.519029 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:34.520266 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ef76abd9-3d0f-43c8-845d-493168028e47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd39b3a1fccad46f9b363fdfdbf642573', 'request_id': 'req-ef76abd9-3d0f-43c8-845d-493168028e47'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:34.529094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ef76abd9-3d0f-43c8-845d-493168028e47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd39b3a1fccad46f9b363fdfdbf642573', 'request_id': 'req-ef76abd9-3d0f-43c8-845d-493168028e47'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:34.535674 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ef76abd9-3d0f-43c8-845d-493168028e47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd39b3a1fccad46f9b363fdfdbf642573', 'request_id': 'req-ef76abd9-3d0f-43c8-845d-493168028e47'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:34.546287 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ef76abd9-3d0f-43c8-845d-493168028e47 admin admin] The token cache is being invalidate because project d39b3a1fccad46f9b363fdfdbf642573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:05:34.546608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ef76abd9-3d0f-43c8-845d-493168028e47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:34.547249 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ef76abd9-3d0f-43c8-845d-493168028e47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:34.548758 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1394/2831] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:05:34 2022] DELETE /identity/v3/projects/d39b3a1fccad46f9b363fdfdbf642573 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:34.552875 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacbd4c-13eb-474d-bb0d-b6a6bf431db8 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:34.554315 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-1eacbd4c-13eb-474d-bb0d-b6a6bf431db8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:34.567736 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacbd4c-13eb-474d-bb0d-b6a6bf431db8 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:34.626014 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7807293-711f-45be-aa43-65db2a5f08ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:34.626589 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7807293-711f-45be-aa43-65db2a5f08ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:34.626873 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7807293-711f-45be-aa43-65db2a5f08ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:34.627149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7807293-711f-45be-aa43-65db2a5f08ad admin admin] PATH_INFO: `/v3/users/a4e0b41d952c476abc419ee384fb51dc` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:34.628312 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7807293-711f-45be-aa43-65db2a5f08ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4e0b41d952c476abc419ee384fb51dc)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:34.630239 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7807293-711f-45be-aa43-65db2a5f08ad admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:34.675620 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f7807293-711f-45be-aa43-65db2a5f08ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4e0b41d952c476abc419ee384fb51dc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:34.682354 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f7807293-711f-45be-aa43-65db2a5f08ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4e0b41d952c476abc419ee384fb51dc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:34.689271 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:34.695823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f7807293-711f-45be-aa43-65db2a5f08ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4e0b41d952c476abc419ee384fb51dc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:34.696829 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f7807293-711f-45be-aa43-65db2a5f08ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4e0b41d952c476abc419ee384fb51dc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:34.697508 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-f7807293-711f-45be-aa43-65db2a5f08ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4e0b41d952c476abc419ee384fb51dc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:34.711468 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1438/2832] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:05:34 2022] DELETE /identity/v3/users/a4e0b41d952c476abc419ee384fb51dc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:34.974712 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76abd9-3d0f-43c8-845d-493168028e47 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:34.976474 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ef76abd9-3d0f-43c8-845d-493168028e47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:34.989593 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76abd9-3d0f-43c8-845d-493168028e47 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:35.045884 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3684907-9255-49bd-9401-ce8cb7be2ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:35.046565 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3684907-9255-49bd-9401-ce8cb7be2ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:35.046857 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3684907-9255-49bd-9401-ce8cb7be2ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:35.047126 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3684907-9255-49bd-9401-ce8cb7be2ae9 admin admin] PATH_INFO: `/v3/projects/216c2acf71d24224af45dce343c1429d` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:35.048271 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3684907-9255-49bd-9401-ce8cb7be2ae9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=216c2acf71d24224af45dce343c1429d)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:35.050164 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3684907-9255-49bd-9401-ce8cb7be2ae9 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:35.094215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c3684907-9255-49bd-9401-ce8cb7be2ae9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '216c2acf71d24224af45dce343c1429d', 'request_id': 'req-c3684907-9255-49bd-9401-ce8cb7be2ae9'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:35.100308 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c3684907-9255-49bd-9401-ce8cb7be2ae9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '216c2acf71d24224af45dce343c1429d', 'request_id': 'req-c3684907-9255-49bd-9401-ce8cb7be2ae9'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:35.106238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c3684907-9255-49bd-9401-ce8cb7be2ae9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '216c2acf71d24224af45dce343c1429d', 'request_id': 'req-c3684907-9255-49bd-9401-ce8cb7be2ae9'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:35.116697 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c3684907-9255-49bd-9401-ce8cb7be2ae9 admin admin] The token cache is being invalidate because project 216c2acf71d24224af45dce343c1429d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80306) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:05:35.116980 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c3684907-9255-49bd-9401-ce8cb7be2ae9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:35.117598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-c3684907-9255-49bd-9401-ce8cb7be2ae9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:35.119092 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1395/2833] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:05:34 2022] DELETE /identity/v3/projects/216c2acf71d24224af45dce343c1429d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:35.124977 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dbb24-ca80-41f5-b6a0-998957be7679 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:35.125446 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dbb24-ca80-41f5-b6a0-998957be7679 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:35.125549 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dbb24-ca80-41f5-b6a0-998957be7679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:35.145548 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-d59dbb24-ca80-41f5-b6a0-998957be7679 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:35.146732 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d59dbb24-ca80-41f5-b6a0-998957be7679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:35.204541 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1439/2834] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 13:05:35.212984 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd05f23-041c-49d4-8be3-99d6cdd6770a None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:35.213268 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd05f23-041c-49d4-8be3-99d6cdd6770a None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:35.213542 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd05f23-041c-49d4-8be3-99d6cdd6770a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:35.233385 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-4cd05f23-041c-49d4-8be3-99d6cdd6770a None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:35.234602 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-4cd05f23-041c-49d4-8be3-99d6cdd6770a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:35.243094 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1396/2835] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:35.247125 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dbb24-ca80-41f5-b6a0-998957be7679 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:35.256775 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dbb24-ca80-41f5-b6a0-998957be7679 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:35.261304 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dd5ab-85e4-413b-baa8-05e8626d6086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:35.261913 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dd5ab-85e4-413b-baa8-05e8626d6086 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:35.262191 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dd5ab-85e4-413b-baa8-05e8626d6086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:35.262502 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dd5ab-85e4-413b-baa8-05e8626d6086 admin admin] PATH_INFO: `/v3/domains` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:35.263215 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4dd5ab-85e4-413b-baa8-05e8626d6086 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80305) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 13:05:35.263521 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4dd5ab-85e4-413b-baa8-05e8626d6086 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:35.265132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4dd5ab-85e4-413b-baa8-05e8626d6086 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:35.276511 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1440/2836] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 13:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 13:05:35.280725 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd05f23-041c-49d4-8be3-99d6cdd6770a None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:35.290151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd05f23-041c-49d4-8be3-99d6cdd6770a None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:35.295054 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53d385-bf10-444e-a2a9-a031d0f90a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:35.295657 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53d385-bf10-444e-a2a9-a031d0f90a95 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:35.295939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53d385-bf10-444e-a2a9-a031d0f90a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:35.296211 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53d385-bf10-444e-a2a9-a031d0f90a95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:35.296879 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53d385-bf10-444e-a2a9-a031d0f90a95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:35.298825 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53d385-bf10-444e-a2a9-a031d0f90a95 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:35.325351 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1397/2837] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:05:35 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:35.329240 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dd5ab-85e4-413b-baa8-05e8626d6086 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:35.339069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dd5ab-85e4-413b-baa8-05e8626d6086 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:35.393866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33279b8b-a9d7-4839-a5d4-2523cdeae8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:35.394470 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33279b8b-a9d7-4839-a5d4-2523cdeae8c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:35.394782 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33279b8b-a9d7-4839-a5d4-2523cdeae8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:35.395055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33279b8b-a9d7-4839-a5d4-2523cdeae8c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:35.395773 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33279b8b-a9d7-4839-a5d4-2523cdeae8c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:35.397668 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33279b8b-a9d7-4839-a5d4-2523cdeae8c7 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:35.418082 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1441/2838] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:05:35 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 13:05:35.422360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53d385-bf10-444e-a2a9-a031d0f90a95 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:35.433306 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53d385-bf10-444e-a2a9-a031d0f90a95 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:35.437932 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176ecc0-41d1-41bd-be10-9cb414440cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:35.438572 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176ecc0-41d1-41bd-be10-9cb414440cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:35.438856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176ecc0-41d1-41bd-be10-9cb414440cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:35.439133 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176ecc0-41d1-41bd-be10-9cb414440cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:35.439829 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176ecc0-41d1-41bd-be10-9cb414440cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:35.441409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d176ecc0-41d1-41bd-be10-9cb414440cd1 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:35.451181 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1398/2839] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:35.454945 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33279b8b-a9d7-4839-a5d4-2523cdeae8c7 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:35.464866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33279b8b-a9d7-4839-a5d4-2523cdeae8c7 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:35.469431 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2734790a-2fa9-4ecb-99e9-0b839c9932c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:35.470036 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734790a-2fa9-4ecb-99e9-0b839c9932c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:35.470318 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734790a-2fa9-4ecb-99e9-0b839c9932c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:35.470624 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734790a-2fa9-4ecb-99e9-0b839c9932c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:35.471319 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734790a-2fa9-4ecb-99e9-0b839c9932c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:35.472895 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2734790a-2fa9-4ecb-99e9-0b839c9932c4 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:35.483819 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1442/2840] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:35.487958 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176ecc0-41d1-41bd-be10-9cb414440cd1 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:35.498316 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176ecc0-41d1-41bd-be10-9cb414440cd1 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:35.502935 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5cba1-f28c-4b66-997c-651c34f344ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:35.503590 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5cba1-f28c-4b66-997c-651c34f344ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:35.503874 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5cba1-f28c-4b66-997c-651c34f344ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:35.504150 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a5cba1-f28c-4b66-997c-651c34f344ea admin admin] PATH_INFO: `/v3/projects/3153146f8746406c8bfe2719a90d7351/users/234e3138d7304df3824c04ec4df4ec83/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:35.519325 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5cba1-f28c-4b66-997c-651c34f344ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=3153146f8746406c8bfe2719a90d7351, user_id=234e3138d7304df3824c04ec4df4ec83, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:35.521938 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a5cba1-f28c-4b66-997c-651c34f344ea admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:35.534547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1399/2841] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:05:35 2022] PUT /identity/v3/projects/3153146f8746406c8bfe2719a90d7351/users/234e3138d7304df3824c04ec4df4ec83/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:35.546456 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deaee36-64a5-4278-9bb4-9237b3a1715b None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:35.546763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deaee36-64a5-4278-9bb4-9237b3a1715b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:35.547035 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deaee36-64a5-4278-9bb4-9237b3a1715b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:35.574281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-4deaee36-64a5-4278-9bb4-9237b3a1715b None None] MFA Rules not processed for user `234e3138d7304df3824c04ec4df4ec83`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:35.575533 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-4deaee36-64a5-4278-9bb4-9237b3a1715b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:35.612452 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:35.640570 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1443/2842] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:35.645116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5cba1-f28c-4b66-997c-651c34f344ea admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:35.650139 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:35.708202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a5cba1-f28c-4b66-997c-651c34f344ea admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:35.713342 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c02709-3472-4609-a286-d999dc2f46d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:35.713951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c02709-3472-4609-a286-d999dc2f46d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:35.714233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c02709-3472-4609-a286-d999dc2f46d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:35.714632 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c02709-3472-4609-a286-d999dc2f46d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:35.715300 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c02709-3472-4609-a286-d999dc2f46d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:35.717169 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c02709-3472-4609-a286-d999dc2f46d1 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:35.743547 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1400/2843] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:05:35 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:35.747678 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deaee36-64a5-4278-9bb4-9237b3a1715b None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:35.758229 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deaee36-64a5-4278-9bb4-9237b3a1715b None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:35.813344 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d7e78-3ba2-468f-9078-84ad090fdc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:35.813947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d7e78-3ba2-468f-9078-84ad090fdc4c admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:35.814229 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d7e78-3ba2-468f-9078-84ad090fdc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:35.814520 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d7e78-3ba2-468f-9078-84ad090fdc4c admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:35.815189 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d7e78-3ba2-468f-9078-84ad090fdc4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:35.817093 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d7e78-3ba2-468f-9078-84ad090fdc4c admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:35.837842 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1444/2844] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:05:35 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 13:05:35.841759 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c02709-3472-4609-a286-d999dc2f46d1 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:35.852410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c02709-3472-4609-a286-d999dc2f46d1 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:35.856987 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff439a0a-659e-4abe-a51f-a61488983b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:35.857590 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff439a0a-659e-4abe-a51f-a61488983b8e admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:35.857867 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff439a0a-659e-4abe-a51f-a61488983b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:35.858138 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff439a0a-659e-4abe-a51f-a61488983b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:35.858860 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff439a0a-659e-4abe-a51f-a61488983b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:35.860444 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff439a0a-659e-4abe-a51f-a61488983b8e admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:35.871223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1401/2845] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:35.875134 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d7e78-3ba2-468f-9078-84ad090fdc4c admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:35.885059 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d7e78-3ba2-468f-9078-84ad090fdc4c admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:35.889640 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed5e43-f2f5-49fb-b9e5-a813eba0f515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:35.890242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed5e43-f2f5-49fb-b9e5-a813eba0f515 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:35.890545 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed5e43-f2f5-49fb-b9e5-a813eba0f515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:35.890819 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed5e43-f2f5-49fb-b9e5-a813eba0f515 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:35.891505 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed5e43-f2f5-49fb-b9e5-a813eba0f515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:35.893079 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed5e43-f2f5-49fb-b9e5-a813eba0f515 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:35.903250 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1445/2846] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:35.907242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff439a0a-659e-4abe-a51f-a61488983b8e admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:35.917949 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff439a0a-659e-4abe-a51f-a61488983b8e admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:35.922419 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee42bab-0063-490f-aa97-a2f63ec2fa2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:35.923088 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee42bab-0063-490f-aa97-a2f63ec2fa2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:35.923363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee42bab-0063-490f-aa97-a2f63ec2fa2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:35.923634 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee42bab-0063-490f-aa97-a2f63ec2fa2e admin admin] PATH_INFO: `/v3/projects/f083feb14baa46d0b1a90ccec7d88529/users/db7beeedf8c343a3a8061e47b16d7d7b/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:35.937584 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee42bab-0063-490f-aa97-a2f63ec2fa2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f083feb14baa46d0b1a90ccec7d88529, user_id=db7beeedf8c343a3a8061e47b16d7d7b, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:35.940231 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee42bab-0063-490f-aa97-a2f63ec2fa2e admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:35.952643 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1402/2847] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:05:35 2022] PUT /identity/v3/projects/f083feb14baa46d0b1a90ccec7d88529/users/db7beeedf8c343a3a8061e47b16d7d7b/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:35.964687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb551e-cfbc-4698-bfae-8a7eba2438fb None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:35.964973 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb551e-cfbc-4698-bfae-8a7eba2438fb None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:35.965249 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb551e-cfbc-4698-bfae-8a7eba2438fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:35.993028 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-73cb551e-cfbc-4698-bfae-8a7eba2438fb None None] MFA Rules not processed for user `db7beeedf8c343a3a8061e47b16d7d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:35.994256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-73cb551e-cfbc-4698-bfae-8a7eba2438fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:36.056286 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1446/2848] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:36.060840 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee42bab-0063-490f-aa97-a2f63ec2fa2e admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:36.120237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee42bab-0063-490f-aa97-a2f63ec2fa2e admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:36.127533 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00aa68d-341b-47e9-ad21-af87dec1f9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:36.128142 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00aa68d-341b-47e9-ad21-af87dec1f9c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:36.128423 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00aa68d-341b-47e9-ad21-af87dec1f9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:36.128698 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00aa68d-341b-47e9-ad21-af87dec1f9c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:36.129355 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00aa68d-341b-47e9-ad21-af87dec1f9c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:36.131273 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00aa68d-341b-47e9-ad21-af87dec1f9c3 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:36.157845 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1403/2849] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:05:36 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:36.161849 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb551e-cfbc-4698-bfae-8a7eba2438fb None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:36.172862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb551e-cfbc-4698-bfae-8a7eba2438fb None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:36.216750 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:36.229948 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7508d-9b04-4683-bce2-241df81fe711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:36.230599 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7508d-9b04-4683-bce2-241df81fe711 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:36.230888 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7508d-9b04-4683-bce2-241df81fe711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:36.231157 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f7508d-9b04-4683-bce2-241df81fe711 admin admin] PATH_INFO: `/v3/users` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:36.231823 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7508d-9b04-4683-bce2-241df81fe711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:36.233756 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f7508d-9b04-4683-bce2-241df81fe711 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:36.253471 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1447/2850] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:05:36 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 13:05:36.258095 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00aa68d-341b-47e9-ad21-af87dec1f9c3 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:36.268273 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00aa68d-341b-47e9-ad21-af87dec1f9c3 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:36.273495 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc69cc6-c695-40c1-b574-cb677bcba889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:36.274095 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc69cc6-c695-40c1-b574-cb677bcba889 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:36.274370 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc69cc6-c695-40c1-b574-cb677bcba889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:36.274682 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc69cc6-c695-40c1-b574-cb677bcba889 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:36.275363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc69cc6-c695-40c1-b574-cb677bcba889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:36.276930 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc69cc6-c695-40c1-b574-cb677bcba889 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:36.287183 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1404/2851] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:36.291137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7508d-9b04-4683-bce2-241df81fe711 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:36.300987 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f7508d-9b04-4683-bce2-241df81fe711 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:36.305465 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397f10a-09a0-453a-b1e0-425ded9a4c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:36.306111 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397f10a-09a0-453a-b1e0-425ded9a4c30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:36.306387 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397f10a-09a0-453a-b1e0-425ded9a4c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:36.306687 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397f10a-09a0-453a-b1e0-425ded9a4c30 admin admin] PATH_INFO: `/v3/projects/7f4b8d797bcc4e83b65f02d493457443/users/8aa5392d6080408caa9a584f0796ac46/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:36.321991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397f10a-09a0-453a-b1e0-425ded9a4c30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f4b8d797bcc4e83b65f02d493457443, user_id=8aa5392d6080408caa9a584f0796ac46, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:36.324677 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397f10a-09a0-453a-b1e0-425ded9a4c30 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:36.337758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1448/2852] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:05:36 2022] PUT /identity/v3/projects/7f4b8d797bcc4e83b65f02d493457443/users/8aa5392d6080408caa9a584f0796ac46/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:36.341630 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc69cc6-c695-40c1-b574-cb677bcba889 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:36.351360 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc69cc6-c695-40c1-b574-cb677bcba889 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:36.374608 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:36.406798 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c08dda-f48c-4e1c-800f-dbbdef416de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:36.407409 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c08dda-f48c-4e1c-800f-dbbdef416de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:36.407691 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c08dda-f48c-4e1c-800f-dbbdef416de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:36.408024 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c08dda-f48c-4e1c-800f-dbbdef416de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:36.408649 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c08dda-f48c-4e1c-800f-dbbdef416de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:36.410221 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c08dda-f48c-4e1c-800f-dbbdef416de2 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:36.419951 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1405/2853] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:36.424115 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397f10a-09a0-453a-b1e0-425ded9a4c30 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:36.434564 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397f10a-09a0-453a-b1e0-425ded9a4c30 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:36.439060 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be2258-fe49-4b71-a722-fe970159d705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:36.439706 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be2258-fe49-4b71-a722-fe970159d705 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:36.439982 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be2258-fe49-4b71-a722-fe970159d705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:36.440252 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5be2258-fe49-4b71-a722-fe970159d705 admin admin] PATH_INFO: `/v3/projects/7f4b8d797bcc4e83b65f02d493457443/users/8aa5392d6080408caa9a584f0796ac46/roles/74ec741f50da4dd1b83c16cb6ecd8446` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:36.442154 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be2258-fe49-4b71-a722-fe970159d705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f4b8d797bcc4e83b65f02d493457443, user_id=8aa5392d6080408caa9a584f0796ac46, role_id=74ec741f50da4dd1b83c16cb6ecd8446)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:36.445432 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5be2258-fe49-4b71-a722-fe970159d705 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:36.455373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1449/2854] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:05:36 2022] PUT /identity/v3/projects/7f4b8d797bcc4e83b65f02d493457443/users/8aa5392d6080408caa9a584f0796ac46/roles/74ec741f50da4dd1b83c16cb6ecd8446 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:36.465183 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36b2c2-39bf-443d-b779-b749e7a6cbbc None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:36.465650 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36b2c2-39bf-443d-b779-b749e7a6cbbc None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:36.465766 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d36b2c2-39bf-443d-b779-b749e7a6cbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:36.492945 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-4d36b2c2-39bf-443d-b779-b749e7a6cbbc None None] MFA Rules not processed for user `8aa5392d6080408caa9a584f0796ac46`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:36.494177 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-4d36b2c2-39bf-443d-b779-b749e7a6cbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:36.559701 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1406/2855] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:36.571746 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be2258-fe49-4b71-a722-fe970159d705 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:36.573429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c5be2258-fe49-4b71-a722-fe970159d705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:36.632560 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5be2258-fe49-4b71-a722-fe970159d705 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:36.636253 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc95b2d-71a4-40f6-b708-a12cd53a5731 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:36.636866 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc95b2d-71a4-40f6-b708-a12cd53a5731 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:36.637140 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc95b2d-71a4-40f6-b708-a12cd53a5731 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:36.637406 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc95b2d-71a4-40f6-b708-a12cd53a5731 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:36.692575 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc95b2d-71a4-40f6-b708-a12cd53a5731 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:36.693922 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc95b2d-71a4-40f6-b708-a12cd53a5731 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:36.703544 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1450/2856] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:38.194774 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36b2c2-39bf-443d-b779-b749e7a6cbbc None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:38.196202 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-4d36b2c2-39bf-443d-b779-b749e7a6cbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:38.208052 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d36b2c2-39bf-443d-b779-b749e7a6cbbc None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:38.211598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c506a5-767d-4cd5-b1b5-00011975ee55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:38.212201 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c506a5-767d-4cd5-b1b5-00011975ee55 service cinder] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:38.212475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c506a5-767d-4cd5-b1b5-00011975ee55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:38.212743 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c506a5-767d-4cd5-b1b5-00011975ee55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:38.268011 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c506a5-767d-4cd5-b1b5-00011975ee55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:38.269426 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c506a5-767d-4cd5-b1b5-00011975ee55 service cinder] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:38.278230 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1407/2857] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:38.966955 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc95b2d-71a4-40f6-b708-a12cd53a5731 service cinder] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:38.976132 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc95b2d-71a4-40f6-b708-a12cd53a5731 service cinder] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:38.979731 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf88be2-c576-43a4-9caa-f9baa173d182 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:38.980328 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf88be2-c576-43a4-9caa-f9baa173d182 service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:38.980603 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf88be2-c576-43a4-9caa-f9baa173d182 service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:38.980900 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf88be2-c576-43a4-9caa-f9baa173d182 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:38.989604 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf88be2-c576-43a4-9caa-f9baa173d182 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:38.991051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf88be2-c576-43a4-9caa-f9baa173d182 service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:39.000376 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1451/2858] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:42.833836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c506a5-767d-4cd5-b1b5-00011975ee55 service cinder] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:42.880788 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=100 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:42.901216 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c506a5-767d-4cd5-b1b5-00011975ee55 service cinder] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:42.905685 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a05ba-67af-4c63-8ccf-b00aaf399b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:42.906315 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a05ba-67af-4c63-8ccf-b00aaf399b1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:42.906627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a05ba-67af-4c63-8ccf-b00aaf399b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:42.906899 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a05ba-67af-4c63-8ccf-b00aaf399b1b admin admin] PATH_INFO: `/v3/users/234e3138d7304df3824c04ec4df4ec83` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:42.908048 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a05ba-67af-4c63-8ccf-b00aaf399b1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=234e3138d7304df3824c04ec4df4ec83)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:42.909934 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a05ba-67af-4c63-8ccf-b00aaf399b1b admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:42.954870 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-406a05ba-67af-4c63-8ccf-b00aaf399b1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '234e3138d7304df3824c04ec4df4ec83'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:42.966439 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-406a05ba-67af-4c63-8ccf-b00aaf399b1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '234e3138d7304df3824c04ec4df4ec83'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:42.973021 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-406a05ba-67af-4c63-8ccf-b00aaf399b1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '234e3138d7304df3824c04ec4df4ec83'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:42.973856 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-406a05ba-67af-4c63-8ccf-b00aaf399b1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '234e3138d7304df3824c04ec4df4ec83'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:42.981194 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-406a05ba-67af-4c63-8ccf-b00aaf399b1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '234e3138d7304df3824c04ec4df4ec83'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:42.986939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1408/2859] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:05:42 2022] DELETE /identity/v3/users/234e3138d7304df3824c04ec4df4ec83 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:42.991397 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269de5e0-5639-4474-b306-1e230b2927c8 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:42.992408 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269de5e0-5639-4474-b306-1e230b2927c8 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:42.992518 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269de5e0-5639-4474-b306-1e230b2927c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:43.010931 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-269de5e0-5639-4474-b306-1e230b2927c8 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:43.012116 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-269de5e0-5639-4474-b306-1e230b2927c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:43.045580 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:43.071398 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1452/2860] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:43.080981 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a05ba-67af-4c63-8ccf-b00aaf399b1b admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:43.082663 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-406a05ba-67af-4c63-8ccf-b00aaf399b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:43.140499 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a05ba-67af-4c63-8ccf-b00aaf399b1b admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:43.144082 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a186e09-3c1c-47b7-8e4f-a3ef910fbb3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:43.144687 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a186e09-3c1c-47b7-8e4f-a3ef910fbb3b service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:43.144963 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a186e09-3c1c-47b7-8e4f-a3ef910fbb3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:43.145237 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a186e09-3c1c-47b7-8e4f-a3ef910fbb3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:43.154941 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a186e09-3c1c-47b7-8e4f-a3ef910fbb3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:43.156365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a186e09-3c1c-47b7-8e4f-a3ef910fbb3b service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:43.166392 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1409/2861] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:43.425578 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269de5e0-5639-4474-b306-1e230b2927c8 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:43.427056 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-269de5e0-5639-4474-b306-1e230b2927c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:43.439974 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269de5e0-5639-4474-b306-1e230b2927c8 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:43.444821 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:43.445473 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:43.445757 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:43.446009 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0 admin admin] PATH_INFO: `/v3/projects/3153146f8746406c8bfe2719a90d7351` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:43.447202 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3153146f8746406c8bfe2719a90d7351)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:43.449165 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:43.496211 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3153146f8746406c8bfe2719a90d7351', 'request_id': 'req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:43.502839 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3153146f8746406c8bfe2719a90d7351', 'request_id': 'req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:43.509412 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3153146f8746406c8bfe2719a90d7351', 'request_id': 'req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:43.519355 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0 admin admin] The token cache is being invalidate because project 3153146f8746406c8bfe2719a90d7351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:05:43.519633 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:43.520268 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:43.521759 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1453/2862] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:05:43 2022] DELETE /identity/v3/projects/3153146f8746406c8bfe2719a90d7351 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:43.526364 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a186e09-3c1c-47b7-8e4f-a3ef910fbb3b service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:43.535964 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a186e09-3c1c-47b7-8e4f-a3ef910fbb3b service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:43.592081 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780c908-7509-4fda-b7b2-2489192eb541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:43.592642 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780c908-7509-4fda-b7b2-2489192eb541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:43.592924 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780c908-7509-4fda-b7b2-2489192eb541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:43.593199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780c908-7509-4fda-b7b2-2489192eb541 admin admin] PATH_INFO: `/v3/users/db7beeedf8c343a3a8061e47b16d7d7b` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:43.594363 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780c908-7509-4fda-b7b2-2489192eb541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db7beeedf8c343a3a8061e47b16d7d7b)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:43.596288 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780c908-7509-4fda-b7b2-2489192eb541 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:43.606215 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:43.641185 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e780c908-7509-4fda-b7b2-2489192eb541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db7beeedf8c343a3a8061e47b16d7d7b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:43.653223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e780c908-7509-4fda-b7b2-2489192eb541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db7beeedf8c343a3a8061e47b16d7d7b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:43.660352 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e780c908-7509-4fda-b7b2-2489192eb541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db7beeedf8c343a3a8061e47b16d7d7b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:43.660993 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e780c908-7509-4fda-b7b2-2489192eb541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db7beeedf8c343a3a8061e47b16d7d7b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:43.668129 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-e780c908-7509-4fda-b7b2-2489192eb541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db7beeedf8c343a3a8061e47b16d7d7b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:43.674464 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1410/2863] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:05:43 2022] DELETE /identity/v3/users/db7beeedf8c343a3a8061e47b16d7d7b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:43.938279 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:43.939746 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:43.959069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d39fd7-ed7b-4480-bb17-0f37597dd6c0 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:44.014890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62245cc4-d7f8-4c7b-be88-ef966ab8292c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:44.015544 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62245cc4-d7f8-4c7b-be88-ef966ab8292c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:44.015788 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62245cc4-d7f8-4c7b-be88-ef966ab8292c admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:44.016063 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62245cc4-d7f8-4c7b-be88-ef966ab8292c admin admin] PATH_INFO: `/v3/projects/f083feb14baa46d0b1a90ccec7d88529` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:44.017223 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62245cc4-d7f8-4c7b-be88-ef966ab8292c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f083feb14baa46d0b1a90ccec7d88529)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:44.019137 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62245cc4-d7f8-4c7b-be88-ef966ab8292c admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:44.063990 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-62245cc4-d7f8-4c7b-be88-ef966ab8292c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f083feb14baa46d0b1a90ccec7d88529', 'request_id': 'req-62245cc4-d7f8-4c7b-be88-ef966ab8292c'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:44.070972 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-62245cc4-d7f8-4c7b-be88-ef966ab8292c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f083feb14baa46d0b1a90ccec7d88529', 'request_id': 'req-62245cc4-d7f8-4c7b-be88-ef966ab8292c'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:44.077240 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-62245cc4-d7f8-4c7b-be88-ef966ab8292c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f083feb14baa46d0b1a90ccec7d88529', 'request_id': 'req-62245cc4-d7f8-4c7b-be88-ef966ab8292c'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:44.081069 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:44.089087 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-62245cc4-d7f8-4c7b-be88-ef966ab8292c admin admin] The token cache is being invalidate because project f083feb14baa46d0b1a90ccec7d88529 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:05:44.089415 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-62245cc4-d7f8-4c7b-be88-ef966ab8292c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:44.090273 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-62245cc4-d7f8-4c7b-be88-ef966ab8292c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:44.091915 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1454/2864] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:05:43 2022] DELETE /identity/v3/projects/f083feb14baa46d0b1a90ccec7d88529 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:44.095867 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780c908-7509-4fda-b7b2-2489192eb541 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:44.097386 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-e780c908-7509-4fda-b7b2-2489192eb541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:44.109838 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780c908-7509-4fda-b7b2-2489192eb541 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:44.164369 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e985dfb-21cc-4b36-947d-f0c6e7fe9ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:44.165002 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e985dfb-21cc-4b36-947d-f0c6e7fe9ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:44.165280 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e985dfb-21cc-4b36-947d-f0c6e7fe9ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:44.165551 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e985dfb-21cc-4b36-947d-f0c6e7fe9ab2 admin admin] PATH_INFO: `/v3/users/8aa5392d6080408caa9a584f0796ac46` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:44.166729 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e985dfb-21cc-4b36-947d-f0c6e7fe9ab2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aa5392d6080408caa9a584f0796ac46)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:44.168880 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e985dfb-21cc-4b36-947d-f0c6e7fe9ab2 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:44.213989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-3e985dfb-21cc-4b36-947d-f0c6e7fe9ab2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aa5392d6080408caa9a584f0796ac46'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:44.226896 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-3e985dfb-21cc-4b36-947d-f0c6e7fe9ab2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aa5392d6080408caa9a584f0796ac46'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:44.234161 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-3e985dfb-21cc-4b36-947d-f0c6e7fe9ab2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aa5392d6080408caa9a584f0796ac46'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:44.234832 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-3e985dfb-21cc-4b36-947d-f0c6e7fe9ab2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aa5392d6080408caa9a584f0796ac46'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:44.242317 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-3e985dfb-21cc-4b36-947d-f0c6e7fe9ab2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aa5392d6080408caa9a584f0796ac46'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:44.248204 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1411/2865] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:05:44 2022] DELETE /identity/v3/users/8aa5392d6080408caa9a584f0796ac46 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:44.518864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62245cc4-d7f8-4c7b-be88-ef966ab8292c admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:44.520300 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-62245cc4-d7f8-4c7b-be88-ef966ab8292c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:44.533685 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62245cc4-d7f8-4c7b-be88-ef966ab8292c admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:44.591490 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94fcab0-cf1b-4100-a192-921b7896687e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:44.592141 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fcab0-cf1b-4100-a192-921b7896687e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:44.592444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fcab0-cf1b-4100-a192-921b7896687e admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:44.592723 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fcab0-cf1b-4100-a192-921b7896687e admin admin] PATH_INFO: `/v3/projects/7f4b8d797bcc4e83b65f02d493457443` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:44.594067 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94fcab0-cf1b-4100-a192-921b7896687e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f4b8d797bcc4e83b65f02d493457443)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:44.596099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94fcab0-cf1b-4100-a192-921b7896687e admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:44.642933 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a94fcab0-cf1b-4100-a192-921b7896687e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f4b8d797bcc4e83b65f02d493457443', 'request_id': 'req-a94fcab0-cf1b-4100-a192-921b7896687e'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:44.650055 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a94fcab0-cf1b-4100-a192-921b7896687e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f4b8d797bcc4e83b65f02d493457443', 'request_id': 'req-a94fcab0-cf1b-4100-a192-921b7896687e'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:44.656188 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a94fcab0-cf1b-4100-a192-921b7896687e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f4b8d797bcc4e83b65f02d493457443', 'request_id': 'req-a94fcab0-cf1b-4100-a192-921b7896687e'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:44.666105 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a94fcab0-cf1b-4100-a192-921b7896687e admin admin] The token cache is being invalidate because project 7f4b8d797bcc4e83b65f02d493457443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:05:44.666390 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a94fcab0-cf1b-4100-a192-921b7896687e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:44.667291 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-a94fcab0-cf1b-4100-a192-921b7896687e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:44.668785 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1455/2866] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:05:44 2022] DELETE /identity/v3/projects/7f4b8d797bcc4e83b65f02d493457443 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:44.674042 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f4c32-460d-48b2-8f2f-cbbd1eab1a4f None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:44.674331 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f4c32-460d-48b2-8f2f-cbbd1eab1a4f None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:44.674715 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f4c32-460d-48b2-8f2f-cbbd1eab1a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:44.688895 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:44.696623 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-2c2f4c32-460d-48b2-8f2f-cbbd1eab1a4f None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:44.698576 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-2c2f4c32-460d-48b2-8f2f-cbbd1eab1a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:44.756061 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1412/2867] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:44.765615 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7292958-6ca7-49ee-bf3c-7daf03e56c3e None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:44.765898 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7292958-6ca7-49ee-bf3c-7daf03e56c3e None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:44.766199 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7292958-6ca7-49ee-bf3c-7daf03e56c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:44.786242 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-b7292958-6ca7-49ee-bf3c-7daf03e56c3e None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:44.787490 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b7292958-6ca7-49ee-bf3c-7daf03e56c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:44.795581 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1456/2868] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:44.799598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f4c32-460d-48b2-8f2f-cbbd1eab1a4f None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:44.810005 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f4c32-460d-48b2-8f2f-cbbd1eab1a4f None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:44.814622 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5f1ca-d527-4566-af19-7b0a1dc4d509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:44.815222 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5f1ca-d527-4566-af19-7b0a1dc4d509 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:44.815504 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5f1ca-d527-4566-af19-7b0a1dc4d509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:44.815771 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5f1ca-d527-4566-af19-7b0a1dc4d509 admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:44.816457 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d5f1ca-d527-4566-af19-7b0a1dc4d509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 13:05:44.816753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d5f1ca-d527-4566-af19-7b0a1dc4d509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:44.818322 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d5f1ca-d527-4566-af19-7b0a1dc4d509 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:44.830328 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1413/2869] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 13:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 13:05:44.834561 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7292958-6ca7-49ee-bf3c-7daf03e56c3e None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:44.843752 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7292958-6ca7-49ee-bf3c-7daf03e56c3e None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:44.848444 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8797d9-eee5-42a3-894c-795b0bea2dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:44.849035 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8797d9-eee5-42a3-894c-795b0bea2dda admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:44.849313 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8797d9-eee5-42a3-894c-795b0bea2dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:44.849584 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8797d9-eee5-42a3-894c-795b0bea2dda admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:44.850253 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8797d9-eee5-42a3-894c-795b0bea2dda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:44.852178 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8797d9-eee5-42a3-894c-795b0bea2dda admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:44.878674 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1457/2870] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:05:44 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:44.883426 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5f1ca-d527-4566-af19-7b0a1dc4d509 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:44.893353 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5f1ca-d527-4566-af19-7b0a1dc4d509 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:44.948268 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972d127-913c-4637-905b-9e16b38f4fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:44.948876 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972d127-913c-4637-905b-9e16b38f4fae admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:44.949160 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972d127-913c-4637-905b-9e16b38f4fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:44.949435 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972d127-913c-4637-905b-9e16b38f4fae admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:44.950108 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972d127-913c-4637-905b-9e16b38f4fae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:44.952073 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972d127-913c-4637-905b-9e16b38f4fae admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:44.973855 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1414/2871] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:05:44 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 13:05:44.978510 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8797d9-eee5-42a3-894c-795b0bea2dda admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:44.988760 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8797d9-eee5-42a3-894c-795b0bea2dda admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:44.993872 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf7598-226e-482b-abc4-6f86ef81519f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:44.994475 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf7598-226e-482b-abc4-6f86ef81519f admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:44.994783 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf7598-226e-482b-abc4-6f86ef81519f admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:44.995057 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf7598-226e-482b-abc4-6f86ef81519f admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:44.995745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf7598-226e-482b-abc4-6f86ef81519f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:44.997335 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaf7598-226e-482b-abc4-6f86ef81519f admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:45.007552 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1458/2872] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:45.011233 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972d127-913c-4637-905b-9e16b38f4fae admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:45.021146 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972d127-913c-4637-905b-9e16b38f4fae admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:45.025762 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10454d8-335d-41a3-b2e0-30fbebd70fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:45.026365 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10454d8-335d-41a3-b2e0-30fbebd70fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:45.026678 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10454d8-335d-41a3-b2e0-30fbebd70fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:45.026957 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10454d8-335d-41a3-b2e0-30fbebd70fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:45.027676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10454d8-335d-41a3-b2e0-30fbebd70fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:45.029303 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10454d8-335d-41a3-b2e0-30fbebd70fb9 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:45.040127 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1415/2873] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 03 13:05:45.044246 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf7598-226e-482b-abc4-6f86ef81519f admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:45.054763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaf7598-226e-482b-abc4-6f86ef81519f admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:45.059451 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da8fc1-ed8d-4c49-90fb-ac11af9fd6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:45.060099 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da8fc1-ed8d-4c49-90fb-ac11af9fd6a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:45.060381 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da8fc1-ed8d-4c49-90fb-ac11af9fd6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:45.060657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da8fc1-ed8d-4c49-90fb-ac11af9fd6a5 admin admin] PATH_INFO: `/v3/projects/138f241d493b4a40a9ba7a56d0c18d25/users/462b374dca06492aa3dc99f8123c805a/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:45.075053 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da8fc1-ed8d-4c49-90fb-ac11af9fd6a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=138f241d493b4a40a9ba7a56d0c18d25, user_id=462b374dca06492aa3dc99f8123c805a, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:45.077676 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da8fc1-ed8d-4c49-90fb-ac11af9fd6a5 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:45.090306 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1459/2874] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:05:45 2022] PUT /identity/v3/projects/138f241d493b4a40a9ba7a56d0c18d25/users/462b374dca06492aa3dc99f8123c805a/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:45.100410 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ce961-9cde-454f-b80a-c28e9c769c50 None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:45.101452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ce961-9cde-454f-b80a-c28e9c769c50 None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:45.101452 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ce961-9cde-454f-b80a-c28e9c769c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:45.128923 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-f66ce961-9cde-454f-b80a-c28e9c769c50 None None] MFA Rules not processed for user `462b374dca06492aa3dc99f8123c805a`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:45.130138 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f66ce961-9cde-454f-b80a-c28e9c769c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:45.191989 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1416/2875] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:45.202909 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da8fc1-ed8d-4c49-90fb-ac11af9fd6a5 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:45.204571 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-30da8fc1-ed8d-4c49-90fb-ac11af9fd6a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:45.225597 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:45.262097 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da8fc1-ed8d-4c49-90fb-ac11af9fd6a5 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:45.265780 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0c035-57cb-43fa-a048-95b291f3abda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '473a5e7e980b4473acec85905fa7078c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:45.266392 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0c035-57cb-43fa-a048-95b291f3abda service cinder] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:45.266702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0c035-57cb-43fa-a048-95b291f3abda service cinder] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:45.266979 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0c035-57cb-43fa-a048-95b291f3abda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:45.275228 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0c035-57cb-43fa-a048-95b291f3abda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:45.276648 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0c035-57cb-43fa-a048-95b291f3abda service cinder] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:45.285587 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1460/2876] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:47.818466 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ce961-9cde-454f-b80a-c28e9c769c50 None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:47.822719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:47.882099 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ce961-9cde-454f-b80a-c28e9c769c50 None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:47.886523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95131c-1f65-4fa2-985c-98d8f91d1847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:47.887157 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95131c-1f65-4fa2-985c-98d8f91d1847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:47.887432 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95131c-1f65-4fa2-985c-98d8f91d1847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:47.887732 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95131c-1f65-4fa2-985c-98d8f91d1847 admin admin] PATH_INFO: `/v3/users/462b374dca06492aa3dc99f8123c805a` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:47.888886 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95131c-1f65-4fa2-985c-98d8f91d1847 admin admin] RBAC: Authorizing `identity:delete_user(user_id=462b374dca06492aa3dc99f8123c805a)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:47.890814 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d95131c-1f65-4fa2-985c-98d8f91d1847 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:47.939116 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1d95131c-1f65-4fa2-985c-98d8f91d1847 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '462b374dca06492aa3dc99f8123c805a'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:47.950563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1d95131c-1f65-4fa2-985c-98d8f91d1847 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '462b374dca06492aa3dc99f8123c805a'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:47.957604 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1d95131c-1f65-4fa2-985c-98d8f91d1847 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '462b374dca06492aa3dc99f8123c805a'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:47.958217 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1d95131c-1f65-4fa2-985c-98d8f91d1847 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '462b374dca06492aa3dc99f8123c805a'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:47.965601 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-1d95131c-1f65-4fa2-985c-98d8f91d1847 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '462b374dca06492aa3dc99f8123c805a'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:47.971408 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1417/2877] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:05:47 2022] DELETE /identity/v3/users/462b374dca06492aa3dc99f8123c805a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:47.976800 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472cbc4-9b99-4591-9c51-f7e4319df5c3 None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:47.977535 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472cbc4-9b99-4591-9c51-f7e4319df5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:47.977535 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472cbc4-9b99-4591-9c51-f7e4319df5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:47.995681 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-8472cbc4-9b99-4591-9c51-f7e4319df5c3 None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:47.996874 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8472cbc4-9b99-4591-9c51-f7e4319df5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:48.054864 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1461/2878] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:48.064427 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95131c-1f65-4fa2-985c-98d8f91d1847 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:48.065818 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-1d95131c-1f65-4fa2-985c-98d8f91d1847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:48.121753 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d95131c-1f65-4fa2-985c-98d8f91d1847 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:48.125347 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a2600-7596-4105-a154-3a39275b599b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:48.125948 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a2600-7596-4105-a154-3a39275b599b service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:48.126223 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a2600-7596-4105-a154-3a39275b599b service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:48.126500 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a2600-7596-4105-a154-3a39275b599b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:48.135835 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a2600-7596-4105-a154-3a39275b599b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:48.137225 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a2600-7596-4105-a154-3a39275b599b service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:48.146888 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1418/2879] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:48.419365 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472cbc4-9b99-4591-9c51-f7e4319df5c3 None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:48.421131 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-8472cbc4-9b99-4591-9c51-f7e4319df5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:48.439797 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472cbc4-9b99-4591-9c51-f7e4319df5c3 None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:48.446328 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cb633-31c4-4989-8ce5-13ac6cbedadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:48.447093 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cb633-31c4-4989-8ce5-13ac6cbedadc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:48.447359 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cb633-31c4-4989-8ce5-13ac6cbedadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:48.447649 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cb633-31c4-4989-8ce5-13ac6cbedadc admin admin] PATH_INFO: `/v3/projects/138f241d493b4a40a9ba7a56d0c18d25` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:48.449050 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cb633-31c4-4989-8ce5-13ac6cbedadc admin admin] RBAC: Authorizing `identity:delete_project(project_id=138f241d493b4a40a9ba7a56d0c18d25)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:48.452186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cb633-31c4-4989-8ce5-13ac6cbedadc admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:48.500894 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-026cb633-31c4-4989-8ce5-13ac6cbedadc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '138f241d493b4a40a9ba7a56d0c18d25', 'request_id': 'req-026cb633-31c4-4989-8ce5-13ac6cbedadc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:48.509335 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-026cb633-31c4-4989-8ce5-13ac6cbedadc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '138f241d493b4a40a9ba7a56d0c18d25', 'request_id': 'req-026cb633-31c4-4989-8ce5-13ac6cbedadc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:48.515884 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-026cb633-31c4-4989-8ce5-13ac6cbedadc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '138f241d493b4a40a9ba7a56d0c18d25', 'request_id': 'req-026cb633-31c4-4989-8ce5-13ac6cbedadc'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:48.526595 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-026cb633-31c4-4989-8ce5-13ac6cbedadc admin admin] The token cache is being invalidate because project 138f241d493b4a40a9ba7a56d0c18d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:05:48.526882 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-026cb633-31c4-4989-8ce5-13ac6cbedadc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:48.527542 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-026cb633-31c4-4989-8ce5-13ac6cbedadc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:48.529051 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1462/2880] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:05:48 2022] DELETE /identity/v3/projects/138f241d493b4a40a9ba7a56d0c18d25 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:48.533913 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4083-8edc-42ca-ac43-c8abe996c1bf None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:48.534220 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4083-8edc-42ca-ac43-c8abe996c1bf None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:48.534507 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4083-8edc-42ca-ac43-c8abe996c1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:48.555078 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-7b9c4083-8edc-42ca-ac43-c8abe996c1bf None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:48.556372 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-7b9c4083-8edc-42ca-ac43-c8abe996c1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:48.575893 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:48.618036 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1419/2881] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:48.625956 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dab94-f66a-4be4-b35a-5046d0bad92c None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:48.626335 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dab94-f66a-4be4-b35a-5046d0bad92c None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:48.626702 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33dab94-f66a-4be4-b35a-5046d0bad92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:48.646429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-b33dab94-f66a-4be4-b35a-5046d0bad92c None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:48.648177 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-b33dab94-f66a-4be4-b35a-5046d0bad92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:48.656300 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1463/2882] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:48.660654 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c4083-8edc-42ca-ac43-c8abe996c1bf None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:48.671448 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c4083-8edc-42ca-ac43-c8abe996c1bf None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:48.676106 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7a7b6-ad3a-41c0-982c-d1f9a2882cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:48.676778 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7a7b6-ad3a-41c0-982c-d1f9a2882cbf admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:48.677142 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7a7b6-ad3a-41c0-982c-d1f9a2882cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:48.677478 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7a7b6-ad3a-41c0-982c-d1f9a2882cbf admin admin] PATH_INFO: `/v3/domains` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:48.678243 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7a7b6-ad3a-41c0-982c-d1f9a2882cbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80306) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 03 13:05:48.678648 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7a7b6-ad3a-41c0-982c-d1f9a2882cbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:48.680797 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7a7b6-ad3a-41c0-982c-d1f9a2882cbf admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:48.692302 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1420/2883] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 3 13:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 03 13:05:48.697226 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33dab94-f66a-4be4-b35a-5046d0bad92c None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:48.707572 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33dab94-f66a-4be4-b35a-5046d0bad92c None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:48.712363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44fc5c-f0fe-4368-aaa2-94c890c60114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:48.712981 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44fc5c-f0fe-4368-aaa2-94c890c60114 admin admin] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:48.713263 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44fc5c-f0fe-4368-aaa2-94c890c60114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:48.713538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44fc5c-f0fe-4368-aaa2-94c890c60114 admin admin] PATH_INFO: `/v3/projects` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:48.714208 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44fc5c-f0fe-4368-aaa2-94c890c60114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:48.716135 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44fc5c-f0fe-4368-aaa2-94c890c60114 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:48.745519 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1464/2884] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 3 13:05:48 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 03 13:05:48.749739 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7a7b6-ad3a-41c0-982c-d1f9a2882cbf admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:48.760317 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7a7b6-ad3a-41c0-982c-d1f9a2882cbf admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:48.820398 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4feb4f-5932-4056-87e1-b31ce3c46452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:48.821110 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4feb4f-5932-4056-87e1-b31ce3c46452 admin admin] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:48.821287 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4feb4f-5932-4056-87e1-b31ce3c46452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:48.821560 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4feb4f-5932-4056-87e1-b31ce3c46452 admin admin] PATH_INFO: `/v3/users` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:48.822242 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4feb4f-5932-4056-87e1-b31ce3c46452 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:48.824199 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4feb4f-5932-4056-87e1-b31ce3c46452 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:48.847728 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1421/2885] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 3 13:05:48 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 03 13:05:48.848973 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44fc5c-f0fe-4368-aaa2-94c890c60114 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:48.861529 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44fc5c-f0fe-4368-aaa2-94c890c60114 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:48.866136 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df657c6-8ff1-4711-b0ec-682257d20ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:48.867348 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df657c6-8ff1-4711-b0ec-682257d20ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:48.867614 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df657c6-8ff1-4711-b0ec-682257d20ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:48.867911 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df657c6-8ff1-4711-b0ec-682257d20ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:48.868745 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df657c6-8ff1-4711-b0ec-682257d20ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:48.870340 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df657c6-8ff1-4711-b0ec-682257d20ca1 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:48.881675 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1465/2886] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 13:05:48.885988 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4feb4f-5932-4056-87e1-b31ce3c46452 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:48.896141 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4feb4f-5932-4056-87e1-b31ce3c46452 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:48.901475 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce60df-eaa7-481e-91a0-9235420d2afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:48.902085 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce60df-eaa7-481e-91a0-9235420d2afb admin admin] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:48.902366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce60df-eaa7-481e-91a0-9235420d2afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:48.902676 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce60df-eaa7-481e-91a0-9235420d2afb admin admin] PATH_INFO: `/v3/roles` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:48.903367 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce60df-eaa7-481e-91a0-9235420d2afb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:48.904959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce60df-eaa7-481e-91a0-9235420d2afb admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:48.919366 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1422/2887] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 3 13:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 03 13:05:48.920528 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df657c6-8ff1-4711-b0ec-682257d20ca1 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:48.931638 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df657c6-8ff1-4711-b0ec-682257d20ca1 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:48.936747 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747df20-b7c6-4539-9160-1b17eaad8e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:48.937420 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747df20-b7c6-4539-9160-1b17eaad8e38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:48.937692 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747df20-b7c6-4539-9160-1b17eaad8e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:48.937968 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747df20-b7c6-4539-9160-1b17eaad8e38 admin admin] PATH_INFO: `/v3/projects/f3689124630045c195cf7110db9e560a/users/e16d6eaf812f41fc8d40f92db192f37b/roles/0f74302099d9402ba2a922f36cff56ef` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:48.940603 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:48.956252 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747df20-b7c6-4539-9160-1b17eaad8e38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3689124630045c195cf7110db9e560a, user_id=e16d6eaf812f41fc8d40f92db192f37b, role_id=0f74302099d9402ba2a922f36cff56ef)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:48.959003 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747df20-b7c6-4539-9160-1b17eaad8e38 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:48.972630 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1466/2888] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 3 13:05:48 2022] PUT /identity/v3/projects/f3689124630045c195cf7110db9e560a/users/e16d6eaf812f41fc8d40f92db192f37b/roles/0f74302099d9402ba2a922f36cff56ef => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 03 13:05:48.983374 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6ab2c-2a23-4a09-91a4-3eddf8e2c53e None None] REQUEST_METHOD: `POST` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:48.983663 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6ab2c-2a23-4a09-91a4-3eddf8e2c53e None None] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:48.983939 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6ab2c-2a23-4a09-91a4-3eddf8e2c53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:49.012043 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.auth.core [None req-d5a6ab2c-2a23-4a09-91a4-3eddf8e2c53e None None] MFA Rules not processed for user `e16d6eaf812f41fc8d40f92db192f37b`. Rule list: `[]` (Enabled: `True`). {{(pid=80306) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:05:49.013174 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d5a6ab2c-2a23-4a09-91a4-3eddf8e2c53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:49.077460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1423/2889] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:05:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 03 13:05:49.091173 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747df20-b7c6-4539-9160-1b17eaad8e38 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:49.092947 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-6747df20-b7c6-4539-9160-1b17eaad8e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:49.174201 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747df20-b7c6-4539-9160-1b17eaad8e38 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:49.178019 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d7eed-d5b3-47a0-8a82-c9f58cd53364 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6066e48aa644a47a889e09cfe204858', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:49.178686 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d7eed-d5b3-47a0-8a82-c9f58cd53364 service glance] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:49.179494 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d7eed-d5b3-47a0-8a82-c9f58cd53364 service glance] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:49.179774 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d7eed-d5b3-47a0-8a82-c9f58cd53364 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:49.189838 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d7eed-d5b3-47a0-8a82-c9f58cd53364 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:49.191462 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d7eed-d5b3-47a0-8a82-c9f58cd53364 service glance] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:49.203880 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1467/2890] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:49.218063 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6ab2c-2a23-4a09-91a4-3eddf8e2c53e None None] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:49.225480 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d5a6ab2c-2a23-4a09-91a4-3eddf8e2c53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:49.256598 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:49.332668 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6ab2c-2a23-4a09-91a4-3eddf8e2c53e None None] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:49.344663 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6319da9-e4b4-49eb-a9ca-71209022f6b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b634f567dca42dc95df175a4f36c1d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:49.345518 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6319da9-e4b4-49eb-a9ca-71209022f6b7 service nova] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:49.345803 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6319da9-e4b4-49eb-a9ca-71209022f6b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:49.346092 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6319da9-e4b4-49eb-a9ca-71209022f6b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:49.348051 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a6319da9-e4b4-49eb-a9ca-71209022f6b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:49.363403 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6319da9-e4b4-49eb-a9ca-71209022f6b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:49.369370 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6319da9-e4b4-49eb-a9ca-71209022f6b7 service nova] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:49.383523 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1424/2891] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:55.980060 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d7eed-d5b3-47a0-8a82-c9f58cd53364 service glance] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:55.981790 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-f52d7eed-d5b3-47a0-8a82-c9f58cd53364 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:56.072617 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d7eed-d5b3-47a0-8a82-c9f58cd53364 service glance] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:56.077813 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b7d4b-725a-4d4b-9ed5-7feeaff2a562 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5094263ac471481db48783b19192e01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:56.078538 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b7d4b-725a-4d4b-9ed5-7feeaff2a562 service placement] REQUEST_METHOD: `GET` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:56.078890 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b7d4b-725a-4d4b-9ed5-7feeaff2a562 service placement] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:56.079256 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761b7d4b-725a-4d4b-9ed5-7feeaff2a562 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:56.081107 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-761b7d4b-725a-4d4b-9ed5-7feeaff2a562 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:56.093758 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b7d4b-725a-4d4b-9ed5-7feeaff2a562 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:56.096703 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761b7d4b-725a-4d4b-9ed5-7feeaff2a562 service placement] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:56.116836 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1468/2892] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:05:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:05:59.231108 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6319da9-e4b4-49eb-a9ca-71209022f6b7 service nova] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:59.235062 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a6319da9-e4b4-49eb-a9ca-71209022f6b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:59.313212 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6319da9-e4b4-49eb-a9ca-71209022f6b7 service nova] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:59.319030 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457ece7-fdf8-40db-9783-f2892333e9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:59.320041 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457ece7-fdf8-40db-9783-f2892333e9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:59.320563 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457ece7-fdf8-40db-9783-f2892333e9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:59.320847 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8457ece7-fdf8-40db-9783-f2892333e9f9 admin admin] PATH_INFO: `/v3/users/9245bd2079df4931846baaf7c3b54e55` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:59.323076 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457ece7-fdf8-40db-9783-f2892333e9f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9245bd2079df4931846baaf7c3b54e55)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:59.324185 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8457ece7-fdf8-40db-9783-f2892333e9f9 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:59.380936 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8457ece7-fdf8-40db-9783-f2892333e9f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9245bd2079df4931846baaf7c3b54e55'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:59.393463 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8457ece7-fdf8-40db-9783-f2892333e9f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9245bd2079df4931846baaf7c3b54e55'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:59.401588 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8457ece7-fdf8-40db-9783-f2892333e9f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9245bd2079df4931846baaf7c3b54e55'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:59.402224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8457ece7-fdf8-40db-9783-f2892333e9f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9245bd2079df4931846baaf7c3b54e55'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:59.410298 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-8457ece7-fdf8-40db-9783-f2892333e9f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9245bd2079df4931846baaf7c3b54e55'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:59.417660 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1425/2893] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:05:59 2022] DELETE /identity/v3/users/9245bd2079df4931846baaf7c3b54e55 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:59.591916 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b7d4b-725a-4d4b-9ed5-7feeaff2a562 service placement] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:59.605861 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761b7d4b-725a-4d4b-9ed5-7feeaff2a562 service placement] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:59.657432 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:59.666043 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cf144-1f86-4110-9da9-277bb91ed400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:59.666731 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cf144-1f86-4110-9da9-277bb91ed400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:59.667275 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cf144-1f86-4110-9da9-277bb91ed400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:59.667545 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28cf144-1f86-4110-9da9-277bb91ed400 admin admin] PATH_INFO: `/v3/projects/4dc6dac8b9044792a15633af68931533` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:59.668719 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28cf144-1f86-4110-9da9-277bb91ed400 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dc6dac8b9044792a15633af68931533)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:59.670630 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28cf144-1f86-4110-9da9-277bb91ed400 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:59.715529 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d28cf144-1f86-4110-9da9-277bb91ed400 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dc6dac8b9044792a15633af68931533', 'request_id': 'req-d28cf144-1f86-4110-9da9-277bb91ed400'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:59.722034 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d28cf144-1f86-4110-9da9-277bb91ed400 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dc6dac8b9044792a15633af68931533', 'request_id': 'req-d28cf144-1f86-4110-9da9-277bb91ed400'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:59.728763 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d28cf144-1f86-4110-9da9-277bb91ed400 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dc6dac8b9044792a15633af68931533', 'request_id': 'req-d28cf144-1f86-4110-9da9-277bb91ed400'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:59.738785 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d28cf144-1f86-4110-9da9-277bb91ed400 admin admin] The token cache is being invalidate because project 4dc6dac8b9044792a15633af68931533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:05:59.739065 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d28cf144-1f86-4110-9da9-277bb91ed400 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:59.739680 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-d28cf144-1f86-4110-9da9-277bb91ed400 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:59.741174 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1469/2894] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:05:59 2022] DELETE /identity/v3/projects/4dc6dac8b9044792a15633af68931533 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:05:59.745151 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457ece7-fdf8-40db-9783-f2892333e9f9 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:05:59.746619 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-8457ece7-fdf8-40db-9783-f2892333e9f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:05:59.759460 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8457ece7-fdf8-40db-9783-f2892333e9f9 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:05:59.778966 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:05:59.816942 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9955ff-20bb-45c8-a493-799f203ab742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:05:59.817573 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9955ff-20bb-45c8-a493-799f203ab742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:05:59.817850 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9955ff-20bb-45c8-a493-799f203ab742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:05:59.818122 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9955ff-20bb-45c8-a493-799f203ab742 admin admin] PATH_INFO: `/v3/users/e9fa409c38ce410aa46d743e6bf017b1` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:05:59.819348 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9955ff-20bb-45c8-a493-799f203ab742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9fa409c38ce410aa46d743e6bf017b1)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:05:59.821301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9955ff-20bb-45c8-a493-799f203ab742 admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:05:59.865521 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ea9955ff-20bb-45c8-a493-799f203ab742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9fa409c38ce410aa46d743e6bf017b1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:59.877719 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ea9955ff-20bb-45c8-a493-799f203ab742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9fa409c38ce410aa46d743e6bf017b1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:59.886155 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ea9955ff-20bb-45c8-a493-799f203ab742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9fa409c38ce410aa46d743e6bf017b1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:59.886959 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ea9955ff-20bb-45c8-a493-799f203ab742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9fa409c38ce410aa46d743e6bf017b1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:59.894044 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-ea9955ff-20bb-45c8-a493-799f203ab742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9fa409c38ce410aa46d743e6bf017b1'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:05:59.900118 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1426/2895] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:05:59 2022] DELETE /identity/v3/users/e9fa409c38ce410aa46d743e6bf017b1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:06:00.067149 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cf144-1f86-4110-9da9-277bb91ed400 admin admin] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:06:00.068632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-d28cf144-1f86-4110-9da9-277bb91ed400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:06:00.082230 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28cf144-1f86-4110-9da9-277bb91ed400 admin admin] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:06:00.139363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a56ab0-8a74-47a8-995c-8109fbb2dc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:06:00.140002 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a56ab0-8a74-47a8-995c-8109fbb2dc7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:06:00.140282 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a56ab0-8a74-47a8-995c-8109fbb2dc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:06:00.140554 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a56ab0-8a74-47a8-995c-8109fbb2dc7d admin admin] PATH_INFO: `/v3/projects/c95d59c048e54d8d805c5f82a5aa78d7` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:06:00.141707 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a56ab0-8a74-47a8-995c-8109fbb2dc7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c95d59c048e54d8d805c5f82a5aa78d7)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:06:00.143639 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a56ab0-8a74-47a8-995c-8109fbb2dc7d admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:06:00.188937 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-60a56ab0-8a74-47a8-995c-8109fbb2dc7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c95d59c048e54d8d805c5f82a5aa78d7', 'request_id': 'req-60a56ab0-8a74-47a8-995c-8109fbb2dc7d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:06:00.196068 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-60a56ab0-8a74-47a8-995c-8109fbb2dc7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c95d59c048e54d8d805c5f82a5aa78d7', 'request_id': 'req-60a56ab0-8a74-47a8-995c-8109fbb2dc7d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:06:00.202305 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-60a56ab0-8a74-47a8-995c-8109fbb2dc7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c95d59c048e54d8d805c5f82a5aa78d7', 'request_id': 'req-60a56ab0-8a74-47a8-995c-8109fbb2dc7d'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:06:00.212363 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-60a56ab0-8a74-47a8-995c-8109fbb2dc7d admin admin] The token cache is being invalidate because project c95d59c048e54d8d805c5f82a5aa78d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:06:00.212643 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-60a56ab0-8a74-47a8-995c-8109fbb2dc7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:06:00.213265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-60a56ab0-8a74-47a8-995c-8109fbb2dc7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:06:00.214798 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1470/2896] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:06:00 2022] DELETE /identity/v3/projects/c95d59c048e54d8d805c5f82a5aa78d7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:06:09.896342 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=42,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:06:10.209352 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=77,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:06:14.082934 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9955ff-20bb-45c8-a493-799f203ab742 admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:06:14.084351 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-ea9955ff-20bb-45c8-a493-799f203ab742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:06:14.096901 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9955ff-20bb-45c8-a493-799f203ab742 admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:06:14.152055 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba36b7-0f56-44d7-80fc-1cf34fb5540a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:06:14.152612 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba36b7-0f56-44d7-80fc-1cf34fb5540a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:06:14.152885 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba36b7-0f56-44d7-80fc-1cf34fb5540a admin admin] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:06:14.153160 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba36b7-0f56-44d7-80fc-1cf34fb5540a admin admin] PATH_INFO: `/v3/users/e16d6eaf812f41fc8d40f92db192f37b` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:06:14.154301 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba36b7-0f56-44d7-80fc-1cf34fb5540a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e16d6eaf812f41fc8d40f92db192f37b)` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:06:14.156219 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba36b7-0f56-44d7-80fc-1cf34fb5540a admin admin] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:06:14.201133 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d7ba36b7-0f56-44d7-80fc-1cf34fb5540a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e16d6eaf812f41fc8d40f92db192f37b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:06:14.212782 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d7ba36b7-0f56-44d7-80fc-1cf34fb5540a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e16d6eaf812f41fc8d40f92db192f37b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:06:14.219189 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d7ba36b7-0f56-44d7-80fc-1cf34fb5540a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e16d6eaf812f41fc8d40f92db192f37b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:06:14.219805 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d7ba36b7-0f56-44d7-80fc-1cf34fb5540a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e16d6eaf812f41fc8d40f92db192f37b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:06:14.227171 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.notifications [None req-d7ba36b7-0f56-44d7-80fc-1cf34fb5540a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e16d6eaf812f41fc8d40f92db192f37b'} {{(pid=80306) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:06:14.232912 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1427/2897] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 3 13:06:14 2022] DELETE /identity/v3/users/e16d6eaf812f41fc8d40f92db192f37b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:06:14.237418 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edd220-a80b-49c0-aa64-e55908c2737b None None] REQUEST_METHOD: `POST` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:06:14.237929 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edd220-a80b-49c0-aa64-e55908c2737b None None] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:06:14.238042 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71edd220-a80b-49c0-aa64-e55908c2737b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:06:14.258023 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.auth.core [None req-71edd220-a80b-49c0-aa64-e55908c2737b None None] MFA Rules not processed for user `f665e42501ed48eda8620c2548069351`. Rule list: `[]` (Enabled: `True`). {{(pid=80305) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 03 13:06:14.259261 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-71edd220-a80b-49c0-aa64-e55908c2737b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:06:14.319281 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1471/2898] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 3 13:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 03 13:06:14.329030 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba36b7-0f56-44d7-80fc-1cf34fb5540a admin admin] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:06:14.330440 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-d7ba36b7-0f56-44d7-80fc-1cf34fb5540a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:06:14.386968 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba36b7-0f56-44d7-80fc-1cf34fb5540a admin admin] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:06:14.390627 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca382e-a990-4d5f-a22c-ab26e5b47cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ff458fd7f74b99a745edbcf13a114b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:06:14.391238 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca382e-a990-4d5f-a22c-ab26e5b47cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:06:14.391519 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca382e-a990-4d5f-a22c-ab26e5b47cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:06:14.391791 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca382e-a990-4d5f-a22c-ab26e5b47cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:06:14.400827 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca382e-a990-4d5f-a22c-ab26e5b47cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:06:14.402318 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca382e-a990-4d5f-a22c-ab26e5b47cf8 service neutron] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:06:14.411716 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1428/2899] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:06:14.685541 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edd220-a80b-49c0-aa64-e55908c2737b None None] Authenticating user token {{(pid=80305) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:06:14.687071 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.fernet_utils [None req-71edd220-a80b-49c0-aa64-e55908c2737b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80305) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:06:14.699830 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71edd220-a80b-49c0-aa64-e55908c2737b None None] Validating token access rules against request {{(pid=80305) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:06:14.704991 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dc3b23-9890-4ec1-b2cd-8858de033d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f665e42501ed48eda8620c2548069351', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '87eaffc3c69e48b2bb1ac115a06dc966', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80305) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:06:14.705632 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc3b23-9890-4ec1-b2cd-8858de033d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:06:14.705914 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc3b23-9890-4ec1-b2cd-8858de033d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:06:14.706186 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dc3b23-9890-4ec1-b2cd-8858de033d88 admin admin] PATH_INFO: `/v3/projects/f3689124630045c195cf7110db9e560a` {{(pid=80305) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:06:14.707373 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc3b23-9890-4ec1-b2cd-8858de033d88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3689124630045c195cf7110db9e560a)` {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:06:14.709265 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dc3b23-9890-4ec1-b2cd-8858de033d88 admin admin] RBAC: Authorization granted {{(pid=80305) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:06:14.756862 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-47dc3b23-9890-4ec1-b2cd-8858de033d88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3689124630045c195cf7110db9e560a', 'request_id': 'req-47dc3b23-9890-4ec1-b2cd-8858de033d88'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:06:14.763755 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-47dc3b23-9890-4ec1-b2cd-8858de033d88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3689124630045c195cf7110db9e560a', 'request_id': 'req-47dc3b23-9890-4ec1-b2cd-8858de033d88'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:06:14.769804 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-47dc3b23-9890-4ec1-b2cd-8858de033d88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3689124630045c195cf7110db9e560a', 'request_id': 'req-47dc3b23-9890-4ec1-b2cd-8858de033d88'} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:06:14.780374 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-47dc3b23-9890-4ec1-b2cd-8858de033d88 admin admin] The token cache is being invalidate because project f3689124630045c195cf7110db9e560a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80305) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 03 13:06:14.780657 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-47dc3b23-9890-4ec1-b2cd-8858de033d88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:06:14.781275 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG keystone.notifications [None req-47dc3b23-9890-4ec1-b2cd-8858de033d88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80305) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 03 13:06:14.782819 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: [pid: 80305|app: 0|req: 1472/2900] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 3 13:06:14 2022] DELETE /identity/v3/projects/f3689124630045c195cf7110db9e560a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 03 13:06:24.407586 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:06:24.777429 dv-cinder-851996-1-iscsi devstack@keystone.service[80305]: DEBUG dbcounter [-] [80305] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80305) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 03 13:07:12.656867 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca382e-a990-4d5f-a22c-ab26e5b47cf8 service neutron] Authenticating user token {{(pid=80306) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 03 13:07:12.658276 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-a2ca382e-a990-4d5f-a22c-ab26e5b47cf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:07:12.726954 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca382e-a990-4d5f-a22c-ab26e5b47cf8 service neutron] Validating token access rules against request {{(pid=80306) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 03 13:07:12.783224 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086294c-6044-4f2f-8599-3f07f7228d9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5094263ac471481db48783b19192e01d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c0774035ad0f41c986771ef4a41c7d60', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80306) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 03 13:07:12.783834 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086294c-6044-4f2f-8599-3f07f7228d9d service placement] REQUEST_METHOD: `GET` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 03 13:07:12.784109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086294c-6044-4f2f-8599-3f07f7228d9d service placement] SCRIPT_NAME: `/identity` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 03 13:07:12.784379 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086294c-6044-4f2f-8599-3f07f7228d9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80306) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 03 13:07:12.786109 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.fernet_utils [None req-f086294c-6044-4f2f-8599-3f07f7228d9d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80306) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 03 13:07:12.797231 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086294c-6044-4f2f-8599-3f07f7228d9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 03 13:07:12.798836 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086294c-6044-4f2f-8599-3f07f7228d9d service placement] RBAC: Authorization granted {{(pid=80306) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 03 13:07:12.808694 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: [pid: 80306|app: 0|req: 1429/2901] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 3 13:07:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 03 13:07:22.804433 dv-cinder-851996-1-iscsi devstack@keystone.service[80306]: DEBUG dbcounter [-] [80306] Writing DB stats keystone:SELECT=58 {{(pid=80306) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}